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(57) Play list information is recorded on the local stor- 
age 200. The play list information contains main-path in- 
formation and sub-path information. The main-path in- 
formation specifies one among a plurality of AVCIips as 
a MainClip and defines a main reproduction section for 
the MainClip. The sub-path information specifies another 
one among the plurality of AVCIips as a SubClip and 
defines, for the SubClip, a sub reproduction section to 



be synchronized with the main reproduction section. On 
the local storage 200, an audio stream specified as the 
SubClip is recorded. The audio stream includes a plural- 
ity of PES packets which are each attached with a packet 
identifier (PID) that includes an upper field and a lower 
field. The upper field indicates that the audio stream con- 
taining the packet is a secondary audio stream whose 
reproduction output is able to be mixed with a reproduc- 
tion output of the primary audio stream. 
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Description 
Technical Field 

5 [0001] The present invention relates to a technical field including the audio mixing technology. 
Background Art 

[0002] The audio mixing technology is a technology for reading, from one or more recording mediums, a primary audio 
to stream containing a primary sound/voice and a secondary audio stream containing a secondary sound/voice, decoding 
the primary and secondary audio streams respectively, combining the decoding results to generate non-compressed 
digital audio, and outputting the non-compressed digital audio. 

The technology enables a plurality of digital streams constituting a motion picture to be provided via different mediums, 
and, for example, makes it possible to attach a commentary of the director to the motion picture, increasing freedom in 
is creating variations of one motion picture. The technology therefore enables a lot of variations to be created from one 
motion picture. 

[0003] The below-identified document, as a prior art, discloses a technology for reading digital streams from the 
recording mediums and reproducing the read digital streams. 
Document 1: Japanese Laid-Open Patent Application No. 2000-228656 

20 

Disclosure of the Invention 

The Problems the Invention Is Going to Solve 

25 [0004] In a reproduction apparatus model that performs such audio mixing, however, there may be a case where while 
the primary audio-stream is being read from an optical disc, the secondary audio stream needs to be read from another 
recording medium. In such a case, it may happen that a TS packet read from the optical disc and a TS packet read from 
the other recording medium are present on the same bus. When this happens, it is not obvious which one or more of a 
plurality of TS packets constitute the primary audio stream or the secondary audio stream. While it is possible to process 

30 the TS packets as a whole, it is difficult to process either one of the TS packets constituting the primary audio stream 
and the TS packets constituting the secondary audio stream. 

[0005] When the TS packets constituting the primary audio stream and the secondary audio stream are processed 
as a whole, and the primary audio stream and the secondary audio stream are output with the same gain, a critical defect 
in quality, such as a breakage of a speaker, may occur if disadvantageous conditions overlap. 
35 it is therefore an object of the present invention to provide a recording medium that can, if TS packets constituting the 
primary audio stream and the secondary audio stream are present on the same bus, process the two types of TS packets 
separately. 

Means to Solve the Problems 

40 

[0006] To solve the above-described problem, the present invention provides a recording medium on which play list 
information and an audio stream are recorded, wherein the play list information contains main-path information and sub- 
path information, the main-path information specifies one among a plurality of digital streams as a main stream and 
defines a main reproduction section for the main stream, the sub-path information specifies another one among the 

45 plurality of digital streams as a sub stream and defines, for the sub stream, a sub reproduction section that is to be 
synchronized with the main reproduction section, the main stream includes a video stream and a primary audio stream 
multiplexed therein, the audio stream is specified as the sub stream and includes a plurality of packets which are each 
attached with a packet identifier that includes an upper field and a lower field, and the upper field indicates that the audio 
stream is a secondary audio stream whose reproduction output is able to be mixed with a reproduction output of the 

50 primary audio stream. 

Effects of the invention 

[0007] With the above-described construction in which the upper field of the packet identifier indicates that the audio 
55 stream is a secondary audio stream whose reproduction output is able to be mixed with a reproduction output of the 
primary audio stream, a demultiplexing unit of a reproduction apparatus can bring in the packets via a bus, refer to the 
upper field of each packet identifier, and according to the indication of the upper field, output the packets to the decoder 
dedicated to Secondary audio streams. 



2 



EP 1 713 075 A1 



[0008] The demultiplexing unit accordingly can separate the PES packets constituting the Secondary audio streams 
by demultiplexing and send the PES packets to a decoder that is not a decoder dedicated to Primary audio streams. 
With such a construction in which packets constituting Primary audio streams and packets constituting Secondary audio 
streams are separated and supplied to different decoders, it is possible to downconvert, downmix, or reduce the gain 
of only the packets that constitute Primary audio streams. This makes it possible to subject only the packets that constitute 
a Primary audio stream to special processes in real time even when reading out a Secondary audio stream from a 
recording medium while reading out the Primary audio stream from an optical disc and supplying the two streams to the 
two respective decoders. Also, the lower field identifies one among a plurality of Secondary audio streams by its serial 
number. With this construction, when a plurality of different languages are available for a commentary, the commentary 
with an appropriate language among these can be reproduced. 



Brief Description of the Drawing 
[0009] 

Fig. 1 shows a use of the recording medium of the present invention. 
Fig. 2 shows the internal structure of the BD-ROM. 

Fig. 3 shows the structure of the file to which the extension "m2ts" is attached. 

Fig. 4 shows what processes the TS packets constituting the AVClip are subjected to before they are written to the 
BD-ROM. 

Fig. 5 shows relationships between the physical unit of the BD-ROM and the Source packets constituting one file 
extent. 

Fig. 6 shows the elementary streams that are multiplexed into the AVClip. 
Fig. 7 shows the internal structure of Clip information. 
Fig. 8 shows EP map settings on a video stream of a motion picture. 
Fig. 9 shows the data structure of the PlayList information. 

Fig. 10 shows the relationships between the AVClip and the PlayList information. 

Fig. 1 1 shows the structure of the file sound. bdmv. 

Fig. 12 shows an internal structure of the local storage 200. 

Fig. 13 shows elementary streams that are multiplexed into the SubClip. 

Fig. 14 shows PID assignment map in the BD-ROM standard. 

Fig. 15A shows the internal structure of the Secondary audio stream. 

Fig. 15B shows an example of the audio frame. 

Fig. 15C shows an internal structure of the metadata. 

Fig. 15D shows an example of the gain control information. 

Fig. 16 shows how the sound volume of the Primary audio stream is controlled by the metadata in the Secondary 
audio stream. 

Fig. 17 shows the data structure of the PlayList information. 

Fig. 18 shows the internal structure of the Subpath information with close-ups. 

Fig. 19 shows the relationships among SubClip in the local storage 200 PlayList information in the local storage 
200, and MainClip in the BD-ROM. 

Fig. 20 shows the EP_map and the Playltem time axis that are set for the MainClip and the EP_map and the 

SubPlayltem time axis that are set for the SubClip composed of Primary and Secondary audio streams. 

Fig. 21 shows the internal structure of the STN_table. 

Fig. 22A shows Stream_attribute corresponding to a video stream. 



[0010] 

Fig. 22B shows Stream_attributes corresponding to the Primary audio stream and the Secondary audio stream. 

Fig. 22C shows Stream_entry for the video streams. 

Fig. 22D shows Sub entries for the Secondary audio streams. 

Fig. 22E shows the internal structure of Comb_info_Secondary_audio_Primary_audio that is associated with a pair 
of Stream_entry and Stream_attribute for the Secondary audio stream. 

Fig. 23 shows the specification of the Primary audio streams by Comb_info_Secondary_audio_Primary audio. 
Fig. 24 shows a virtual package that is generated by the reproduction apparatus 300. 
Fig. 25 shows the internal structure of the reproduction apparatus of the present invention. 
Fig. 26 shows the functions of the controller 22. 
Fig. 27A shows the bit assignment in the PSR1. 
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[0011] 



Fig. 
Fig. 
Fig. 
Fig. 



27B shows the bit assignment in the PSR14. 
27C shows the bit assignment in the PSR31 . 

28 is a flowchart showing the PlayList reproduction procedure by the PlayList processing unit. 
29A shows the status transition of the value set in PSR1 . 



[0012] 



Fig. 
Fig. 
Fig. 
Fig. 



29B is a flowchart of the "Procedure when playback condition is changed" in PSR1 . 

30 is a flowchart showing the detailed procedure of step S5. 

31 is a flowchart showing the procedure for setting PSR1 when stream change is requested. 
32A shows the status transition of the value set in PSR14. 



[0013] 



Fig. 32B is a flowchart of the "Procedure when playback condition is changed" in PSR1 4. 
Fig. 33 is a flowchart showing the detailed procedure of step S35. 

Fig. 34 is a flowchart showing the procedure for setting PSR14 when stream change is requested. 

Fig. 35A shows the connection form of the reproduction apparatus 300, the AV amplifier 500, and the speakers 600. 



Fig. 35B shows how the number of channels changes in each of the reproduction apparatus 300, the AV amplifier 
500, and the speakers 600. 
Fig. 36A shows the DIB. 

Fig. 36B shows the values that can be set in each field of the DIB. 

Fig. 36C shows a communication sequence using the DIB between the reproduction apparatus 300 and the AV 
amplifier 500. 

Fig. 37 shows an internal structure of PSR15 in the PSR set 23. 

Fig. 38 is a flowchart showing the processing procedure of the controller 22 in the second embodiment. 

Fig. 39 shows a menu for receiving a determination on whether or not to perform the pass-through output. 

Fig. 40 shows a menu for receiving a determination on whether or not to perform the pass-through output. 

Fig. 41 is a flowchart showing the procedures for setting Player capability for Audio in PSR1 5 according to the DIB. 

Fig. 42 is a flowchart showing the procedures for setting Player capability for Audio in PSR1 5 according to the DIB. 

Fig. 43A shows an example of the reproduction apparatus 300 mounted in a car. 



Fig. 43B shows the internal structure of PSR31 in the third embodiment. 

Fig. 44 shows the internal structure of the PlayList Mark information contained in the PlayList information. 

Fig. 45 shows a specification of a chapter position by the PL Mark information. 

Fig. 46 is a flowchart of a processing procedure of a Java ™ application in the third embodiment. 

Fig. 47 is a flowchart showing the procedure of the chapter search. 

Fig. 48 is a flowchart showing the procedure of the chapter skip. 

Fig. 49 is a flowchart of a Java ™ application in the fourth embodiment. 

Fig. 50 shows an example of displaying a menu having an "update button". 

Description of Characters 



[0014] 



[0015] 



[0016] 



100 
200 
300 
400 
500 
600 



BD-ROM 

local storage 

reproduction apparatus 

television 

AV amplifier 

speakers 
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1a BD-ROM drive 

I b bus 

2a, 2b read buffers 

3a, 3b demultiplexers 

5 4 video decoder 

5 video plane 

6a, 6b buffers 

7a, 7b audio decoders 

8 DownMix/DownSample 

10 9a, 9b mixers 

1 0a switch 

10b encoder 

I I Interactive Graphics decoder 
12 Interactive Graphics plane 

15 13 Presentation Graphics decoder 

14 Presentation Graphics plane 

15 JPEG decoder 

16 Still plane 

17 combining unit 

20 1 8a, 1 8b STC generating units 

19a, 19b ATC generating units 

21 memory 

22 controller 

23 PSR set 

25 24 PI D conversion unit 

25 communication unit 

26 operation receiving unit 

27 HDMI transmission/reception unit 
41 PlayList processing unit 

30 42 Procedure executing unit 

43 Procedure executing unit 

44 mixing control unit 



Best Mode for Carrying Out the Invention 

35 

First Embodiment 

[0017] The following describes an embodiment of the recording medium of the present invention. First, the use of the 
recording medium of the present invention will be described. Fig. 1 shows a use of the recording medium of the present 
40 invention. In Fig. 1 , the recording medium of the present invention is a local storage 200. The local storage 200 is used 
for the purpose of supplying a motion picture to a home theater system composed of a reproduction apparatus 300, a 
television 400, an AV amplifier 500, and speakers 600. 

[0018] The BD-ROM 100, the local storage 200, and the reproduction apparatus 300 will now be described. 
The BD-ROM 100 is a recording medium in which a motion picture is recorded. 
45 The local storage 200 is a hard disk that is embedded in the reproduction apparatus, and is used for storing a content 
that is distributed from a server of a motion picture distributor. 

[001 9] The reproduction apparatus 300 is a digital home electric appliance supported for networks, and has a function 

to reproduce a content of the BD-ROM 100. The reproduction apparatus 300 is also able to combine a content, which 

is downloaded from the server of the motion picture distributor via a network, with the content recorded on the BD-ROM 
50 1 00 to expand/update the content of the BD-ROM 100. With a technology called "virtual package", it is possible to 

combine a content recorded on the BD-ROM 100 with a content recorded on the local storage 200 and treat the data, 

which is not recorded on the BD-ROM 100, as if it is recorded on the BD-ROM 100. 

[0020] Up to now, the use of the recording medium of the present invention has been described. 

From now on, the production of the recording medium of the present invention will be described. The recording medium 
55 of the present invention can be achieved by an improvement on a file system of the BD-ROM. Fig. 2 shows a file/directory 

structure of the BD-ROM. In Fig. 2, file/directory structure of the BD-ROM includes a BDMV directory under a root directory. 

[0021] <BD-ROM Outline> 

Fig. 2 shows an internal structure of the BD-ROM. The fourth row of Fig. 2 shows the BD-ROM. The third row shows 
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tracks on the BD-ROM. Fig. 2 shows the tracks in the state where they are horizontally extended though they are in 
reality formed spirally in order from the inner circumference to the outer circumference. The tracks include a lead-in 
area, a volume area, and a lead-out area. The volume area of Fig. 2 has a layer model that includes a physical layer, a 
file system layer, and an application layer. The 1 st row of Fig. 2 shows an application layer format (application format) 
5 of the BD-ROM represented by a directory structure. In the 1 st row of Fig. 2, the BD-ROM includes a BDMV directory 
under a root directory. 

[0022] The BDMV directory includes files to which an extension "bdmv" is attached ("index. bdmv" and "Movie Ob- 
ject, bdmv"). Under the BDMV directory, there are six subdirectories: PLAYLIST directory, CLIPINF directory, STREAM 
directory, BDBJ directory, BDJA directory, and AUXDATA directory. 

10 The PLAYLIST directory includes a file to which an extension "mpls" is attached ("00001 .mpls"). 

[0023] The CLIPINF directory includes a file to which an extension "dpi" is attached ("00001 .dpi"). 
The STREAM directory includes a file to which an extension "m2ts" is attached ("00001 .m2ts"). 
The BDBJ directory includes a file to which an extension "bobj" is attached ("00001 .bobj"). 
The BDJA directory includes a file to which an extension "jar" is attached ("00001 .jar"). 

is The AUXDATA directory includes a file "sound. bdmv". 

[0024] It is understood from the above-described directory structure that a plurality of different types of files are stored 
in the BD-ROM. 

<BD-ROM Structure 1 : AVCIip> 

Firstly, files with the extension "m2ts" will be explained. Fig. 3 shows the structure of the file to which the extension 

20 "m2ts" is attached. The file to which the extension "m2ts" is attached ("00001 .m2ts") stores an AVCIip. The AVClip is a 
digital stream in the MPEG2-Transport Stream format. The digital stream is generated by converting the digitized video 
and audio (upper 1 st row) into an elementary stream composed of PES packets (upper 2 nd row), and converting the 
elementary stream into TS packets (upper 3 rd row), and similarly, converting the Presentation Graphics (PG) stream for 
the subtitles or the like and the Interactive Graphics (IG) stream for the interactive purposes (lower 1 st row, lower 2nd 

25 row) into the TS packets (3 rd row), and then finally multiplexing these TS packets. 

[0025] The PG stream is a graphics stream constituting a subtitle written in a language. There are a plurality of streams 
that respectively correspond to a plurality of languages such as English, Japanese, and French. The PG stream is 
composed of functional segments such as: PCS (Presentation Control Segment) ; PDS (Pallet Define Segment) ; WDS 
(Window Define Segment) ; ODS (Object Define Segment) ; and END (END of Display Set Segment). The ODS (Object 

30 Define Segment) is a functional segment that defines a graphics object as a subtitle. 

[0026] The WDS is a functional segment that defines a drawing area of a graphics obj ect on the screen. The PDS 
(Pallet Define Segment) is a functional segment that defines a color in drawing a graphics object. The PCS is a functional 
segment that defines a page control in displaying a subtitle. Such page control includes Cut-In/Out, Fade-In/Out, Color 
Change, Scroll, and Wipe-In/Out. It is possible with the page control by the PCS to achieve a display effect, for example, 

35 fading out the current subtitle while displaying the next subtitle. 

[0027] The IG stream is a graphics stream for achieving an interactive control. The interactive control defined by the 
IG stream is an interactive control that is compatible with an interactive control on a DVD reproduction apparatus. The 
IG stream is composed of functional segments such as: ICS (Interactive Composition Segment); PDS (Palette Definition 
Segment); CDS (Object Definition Segment) ; and END (END of Display Set Segment) . The ODS (Object Definition 

40 Segment) is a functional segment that defines a graphics object. A button on the interactive screen is drawn by a plurality 
of such graphics objects. The PDS (Palette Definition Segment) is a functional segment that defines a color in drawing 
a graphics object. The ICS (Interactive Composition Segment) is a functional segment that achieves a state change in 
which the button state changes in accordance with a user operation. The ICS includes a button command that is executed 
when a confirmation operation is performed on a button. 

45 [0028] The AVCIip is composed of one or more "STC_Seuences". The "STC_Seuence" is a section that does not 
include a system time-base discontinuity of the System Time Clock (STC), which is the standard time of the system for 
the AV stream. The system time-base discontinuity of the STC is a point at which the disco ntinuity_indicator of the PCR 
packet, which carries the PCR (Program Clock Reference) that is referred to by the decoder to obtain the STC, is ON. 
Next, how the AVCIip having the above-described construction is written to the BD-ROM will be explained. Fig. 4 shows 

50 what processes the TS packets constituting the AVCIip are subjected to before they are written to the BD-ROM. The 1 st 
row of Fig. 4 shows the TS packets constituting the AVCIip. 

[0029] As shown in the 2 nd row of Fig. 4, a 4-byte TS_extra_header (shaded portions in the drawing) is attached to 
each 188-byte TS packet constituting the AVCIip to generate each 192-byte Source packet. The TS_extra_header 
includes Arrival_Time_Stamp that is information indicating the time at which the TS packet is input to the decoder. 
55 The AVCIip shown in the 3 rd row includes one or more "ATC_Sequences" each of which is a sequence of Source packets. 
The "ATC_Sequence" is a sequence of Source packets, where Arrival_Time_Clocks referred to by the Arrival_Time_ 
Stamps included in the ATC Sequence do not include "arrival time-base discontinuity". In other words, the "ATC_ 
Sequence" is a sequence of Source packets, where Arrival Time Clocks referred to by the Arrival Time Stamps included 
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in the ATC_Sequence are continuous. 

[0030] Such_ATC_Sequences constitute the AVClip, and is recorded on the BD-ROM with a file name "xxxxx.m2ts". 
The AVClip is, as is the case with the normal computer files, divided into one or more file extents, which are then recorded 
in areas on the BD-ROM. The 4 th row shows how the AVClip is recorded on the BD-ROM. In the 4 th row, each file extent 
constituting the file has a data length that is equal to or larger than a predetermined length called Sextent. 
[0031] Sextent is the minimum data length of each file extent, where an AVClip is divided into a plurality of file extents 
to be recorded. 

It is presumed here that the time required for the optical pickup to jump to a location on the BD-ROM is obtained by the 
following equation: 

Tjump = Taccess + Toverhead. 

The "Taccess" is a time required that corresponds to a jump distance (a distance to a jump-destination physical 
address). 

The TS packets read out from the BD-ROM are stored in a buffer called the read buffer, and then output to the decoder. 
The "Toverhead" is obtained by the following equation when the input to the read buffer is performed with a bit rate 
called the Rud and the number of sectors in the ECC block is represented by Secc: 

Toverhead ^ (2 X Secc X 8) / Rud = 2 0msec. 

TS packets read out from the BD-ROM are stored in the read buffer in the state of Source packets, and then supplied 
to the decoder at a transfer rate called "TS_Recording_rate". 

[0032] To keep the transfer rate of the TS_Recording_rate while the TS packets are supplied to the decoder, it is 
necessary that during Tjump, the TS packets are continuously output from the read buffer to the decoder. Here, Source 
packets, not TS packets, are output from the read buffer. As a result, when the ratio of the TS packet to the Source 
packet in size is 192/188, it is necessary that during Tjump, the Source packets are continuously output from the read 
buffer at a transfer rate of "1 92/1 88 x TS_Recording_rate". 

Accordingly, the amount of occupied buffer capacity of the read buffer that does not cause an underflow is represented 
by the following equation: 

Boccupied S (Tjump / 1000 X 8) X ((192/188) X 
TS_Recording_rate) . 

The input rate to the read buffer is represented by Rud, and the output rate from the read buffer is represented by TS_ 
Recording_rate x (192/188). Therefore, the occupation rate of the read buffer is obtained by performing" (input rate) - 
(output rate)", and thus obtained by "(Rud - TS_Recording rate) x (192/188)". 

[0033] The time "Tx" required to occupy the read buffer by "Boccupied" is obtained by the following equation: 



Tx = Boccupied / (Rud — TS_Recording_rate X (192/188)) . 

When reading from the BD-ROM, it is necessary to continue to input TS packets with the bit rate Rud for the time period 
"Tx". As a result, the minimum data length Sextent per extent when the AVClip is divided into a plurality of file extents 
to be recorded is obtained by the following equations: 
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S extent = RudXTx 

= RudX Boccupied/ (Rud~ TS_Recording_rate X (192/188) ) 
^ RudX (Tjump/1000X8) X ( (192/188) X TS_Recording__rate ) 

/ (Rud-TS_Recording__rateX (192/188) ) 
^ (RudXTjump/1000X8) X TS_Recording_rate X 1 92 

/ (Rud X 1 8 8 - TS_Recording_rate X 1 92 ) . 

Hence, 

Sextent ^ ( T j ump X Rud/ 10 0 0X8) X 

(TS_Recording_rateXl92/ (RudX 188 — TS_Recording_rate 
X192) ) . 

If each file extent constituting the AVClip has the data length that is equal to or larger than Sextent that is calculated as 
a value that does not cause an underflow of the decoder, even if the file extents constituting the AVClip are located 
discretely on the BD-ROM, TS packets are continuously supplied to the decoder so that the data is read out continuously 
during reproduction. 

[0034] Fig. 5 shows relationships between the physical unit of the BD-ROM and the Source packets constituting one 
file extent. As shown in the 2 nd row, a plurality of sectors are formed on the BD-ROM. The Source packets constituting 
the file extent are, as shown in the 1 st row, divided into groups each of which is composed of 32 Source packets. Each 
group of Source packets is then written into a set of three consecutive sectors. The group of 32 Source packets is 6144 
bytes (= 32 x 1 92), which is equivalent to the size of three sectors (= 2048 x 3). The 32 Source packets stored in the 
three sectors is called an "Aligned Unit". Writing to the BD-ROM is performed in units of Aligned Units. 
[0035] In the 3 rd row, an error correction code is attached to each block of 32 sectors. The block with the error correction 
code is referred to as an ECC block. In so far as it access the BD-ROM in units of Aligned Units, the reproduction 
apparatus can obtain 32 complete Source packets. Up to now, writing process of the AVClip to the BD-ROM has been 
described. 

<Types of Elementary Streams> 

Fig. 6 shows the elementary streams that are multiplexed into the AVClip. 

[0036] As shown in Fig. 6, multiplexed into the AVClip are: a high-image-quality video stream having PID of 0x101 1 ; 
Primary audio streams having PIDs of 0x1 1 00 to 0x1 1 1 F; PG streams having PIDs of 0x1200 to 0x121 F; and IG streams 
having PIDs of 0x1400 to 0x141 F. The packets constituting the elementary streams are assigned with corresponding 
PIDs, and demultiplexed based on the PIDs. Hereinafter, an AVClip in which such a high-image-quality video stream is 
multiplexed is referred to as "MainClip", and an AVClip that is reproduced simultaneously with the MainClip is referred 
to as "SubClip". 

<BD-ROM Structure 2: Clip lnformation> 

Next, files to which an extension "dpi" is attached will be described. A file (00001. cipi) to which an extension "dpi" is 
attached stores Clip information. The Clip information is management information on each AVClip. Fig. 7 shows the 
internal structure of Clip information. As shown on the left-hand side of the drawing, the Clip information includes: 

i) "Clipinfo()" storing information regarding the AVClip; 

ii) "Sequence lnfo()" storing information regarding the ATC Sequence and the STC Sequence; 

iii) "Program lnfo()" storing information regarding the Program Sequence; and 

iv) "Characteristic Point Info (CPI())". 

[0037] The "Cliplnfo" includes "application_type" indicating the application type of the AVClip referred to by the Clip 
information. The "TS_recording_rate", which indicates whether or not video is contained or whether or not still pictures 
(slide show) are contained, is system bit rate information of the AVClip. 
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The Sequence Info is information regarding one or more STC-Sequences and ATC-Sequences contained in the AVCIip. 
The reason that these information are provided is to preliminarily notify the reproduction apparatus of the system time- 
base discontinuity and the arrival time-base discontinuity. That is to say, if such discontinuity is present, there is a 
possibility that a PTS and an ATS that have the same value appear in the AVCIip. This might be a cause of a defective 
reproduction. The Sequence Info is provided to indicate from where to where in the transport stream the STCs or the 
ATCs are sequential. 

[0038] The Program Info is information that indicates a section (called "Program Sequence") of the program where 
the contents are constant. Here, "Program" is a group of elementary streams that have in common a time axis for 
synchronized reproduction. The reason that the Program info is provided is to preliminarily notify the reproduction 
apparatus of a point at which the Program contents changes. It should be noted here that the point at which the Program 
contents changes is, for example, a point at which the PID of the video stream changes, or a point at which the type of 
the video stream changes from SDTV to HDTV. 

From now on, the Characteristic Point Info will be described. The lead line cu2 indicates the close-up of the structure of 
CPI. As indicated by the lead line cu2, the CPI is composed of the Ne number of EP_map_for_one_stream_PIDs: EP_ 
map_for_one_stream_PID [0] ... EP_map_for_one_stream_PID [Ne-1]). These EP_map_for_one_stream_PIDs are 
EP_maps of the elementary streams that belong to the AVCIip. The EP_map is information that indicates, in association 
with an entry time (PTS_EP_start), a packet number (SPN_EP_start) at an entry position where the Access Unit is 
present in one elementary stream. The lead line cu3 in the drawing indicates the close-up of the internal structure of 
EP_map_for_one_stream_PI D. 

[0039] It is understood from this that the EP_map_for_one_stream_PID is composed of the Ne number of EP_Highs 
(EP_High(0) ... EP_High(Nc-1 )) and the Nf number of EP_Lows (EP_Low(0) ... EP_Low(Nf-1 )). Here, the EP High plays 
a role of indicating upper bits of the SPN_EP_start and the PTS_EP_start of the Access Unit (Non-IDR l-Picture, IDR- 
Picture), the EP Low plays a role of indicating lower bits of the SPN EP start and the PTS_EP_start of the Access Unit 
(Non-IDR l-Picture, IDR-Picture). 

[0040] The lead line cu4 in the drawing indicates the close-up of the internal structure of EP_High. As indicated by 
the lead line cu4, the EP High (i) is composed of: "ref_to_EP_Low_id[i]" that is a reference value to EP_Low; "PTS_EP_ 
Highp]" that indicates upper bits of the PTS of the Access Unit (Non-IDR 1 -Picture, IDR-Picture); and "SPN_EP_High 
[i]" that indicates upper bits of the SPN of the Access Unit (Non-IDR l-Picture, IDR-Picture). Here, "i" is an identifier of 
a given EP_High. 

[0041] The lead line cu5 in the drawing indicates the close-up of the structure of EP_Low. As indicated by the lead 
line cu5, the EP_Low (i) is composed of: "is_angle_change_point (EP_Low_id)" that indicates whether or not the cor- 
responding Access Unit is an IDR picture; "l_end_position_offset (EP_Low_id)" that indicates the size of the correspond- 
ing Access Unit; "PTS_EP_Low(EP_Low_id)" that indicates lower bits of the PTS of the Access Unit (Non-IDR l-Picture, 
IDR-Picture); and "SPN EP Low (EP Low id) "that indicates lower bits of the SPN of the Access Unit (Non-IDR l-Picture, 
IDR-Picture). Here, "EP_Low_id" is an identifier for identifying a given EP_Low. 
[0042] <Clip Information Explanation 2: EP_Map> 

Here, the EP_map will be explained in a specific example. Fig. 8 shows EP_map settings on a video stream of a motion 
picture. The 1 st row shows a plurality of pictures (IDR picture, l-Picture, B-Picture, and P-Picture defined in MPEG4- 
AVC) . The 2 nd row shows the time axis for the pictures. The 4 th row indicates a packet sequence, and the 3 rd row 
indicates settings of the EP_map. 

[0043] It is presumed here that in the time axis of the 2 nd row, an IDR picture or an l-Picture is present at each time 
point t1 ... t7. The interval between adjacent ones of the time point t1 ... t7 is approximately one second. The EP_map 
used for the motion picture is set to indicate t1 to t7 as the entry times (PTS_EP_start), and indicate entry positions 
(SPN_EP_start) in association with the entry times. 
<PlayList lnformation> 

Next, the PlayList information will be described. A file (00001 .mpls) to which extension "mpls" is attached is a file storing 
the PlayList (PL) information. 

[0044] Fig. 9 shows the data structure of the PlayList information. As indicated by the lead line mp1 in Fig. 9, the 
PlayList information includes: MainPath information (Main Path ()) that defines MainPath; and PlayListMark information 
(PlayListMark()) that defines chapter. 

<PlayList Information Explanation 1: MainPath lnformation> 

First, the MainPath will be described. The MainPath is a presentation path that is defined in terms of the video stream 
as the main image and the audio stream. 

[0045] As indicated by the arrow mplm the MainPath is defined by a plurality of pieces of Playltem information: Playltem 
information #1 ... Playltem information #m. The Playltem information defines one or more logical reproduction sections 
that constitute the MainPath. The lead line hs1 in the drawing indicates the close-up of the structure of the Playltem 
information. As indicated by the lead line hs1 , the Flayltem information is composed of:"Clip_lnformation_file_name"that 
indicates the file name of the reproduction section information of the AVCIip to which the IN point and the OUT point of 
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the reproduction section belong; "Clip_codec_identifier" that indicates the AVClip encoding method; "is_multi_angle" 
that indicates whetherornotthe Playltem is multi angle; "connection_condition" that indicates whether or not to seamlessly 
connect the current Playltem and the preceding Playltem; "ref_to_STC_id[0]" that indicates uniquely the STC_Sequence 
targeted by the Playltem; "ln_time" that is time information indicating the start point of the reproduction section; "Out_ 

5 time" that is time information indicating the end point of the reproduction section; "UO_mask_table" that indicates which 
user operation should be masked by the Playltem; "Playltem_random_access_flag" that indicates whether or not to 
permit a random access to a mid-point in the Playltem; "Still_mode" that indicates whether or not to continue a still 
display of the last picture after the reproduction of the Playltem ends; and "STN_table". Among these, the time information 
"ln_time" indicating the start point of the reproduction section and the time information "Out_time" indicating the end 

10 point of the reproduction section constitute a presentation path. The presentation path information is composed of "ln_ 
time" and "Out_time". 

[0046] Fig. 10 shows the relationships between the AVClip and the PlayList information. The 1 st row shows the time 
axis held by the PlayList information. The 2 nd to 5 th rows show the video stream that is referenced by the EP_map. 
The PlayList information includes two pieces of Playltem information: Playltem information #1 ; and Playltem information 
is #2. Two reproduction sections are defined by "In time" and "Out_time" included in the Playltem information #1 and 
Playltem information #2. When these reproduction sections are arranged, a time axis that is different from the AVClip 
time axis is defined. This is the PlayList time axis shown in the 1 st row. As understood from this, it is possible, by defining 
the Playltem information, to define a presentation path that is different from the AVClip. 

The above-described Clip information and PlayList information are classified into "static scenario". This is because the 
20 Clip information and the PlayList information define a PlayList that is a static reproduction unit. This completes the 
description of the static scenario. 

[0047] From now on, the "dynamic scenario" will be explained. The dynamic scenario is scenario data that dynamically 
defines the reproduction control of the AVClip. Here, "dynamically" indicates that the contents of the reproduction control 
change due to state change in the reproduction apparatus or due to a key event from the user. The BD-ROMpresumes 

25 two modes as operation environments for the reproduction control. The first operation environment resembles that of 
the DVD reproduction apparatus, and is a command-based execution environment. The other is an operation environment 
of the Java ™ virtual machine. The first operation environment is called the HDMV mode. The second operation envi- 
ronment is called the BD-J mode. Since these two operation environments are available, the dynamic scenario is written 
to conform to one of these operation environments. A dynamic scenario written to conform to the HDMV mode is called 

30 "Movie Object", and a dynamic scenario written to conform to the BD-J mode is called "BD-J Object". 
[0048] First, the Movie Object will be described. 
<Movie Object> 

The Movie Object is stored in a file "MovieObject.bdmv" shown in Fig. 2 and includes a navigation command sequence. 
The navigation command sequence is a sequence of navigation commands that achieve a conditional branch, setting 
35 the status register in the reproduction apparatus, acquiring a value set in the status register, and so on. The following 
navigation commands can be written in the Movie Object. 
PlayPL Command 

Format: PlayPL(1 st argument, 2 nd argument) 

As the 1 st argument, a PlayList number is written to specify a PlayList to be reproduced. As the 2 nd argument, a Playltem 
^o included in the PlayList, given time in the PlayList, Chapter, or Mark to specify a reproduction start position. 

[0049] A PlayPL function using the Playltem to specify a reproduction start position in the PL time axis is referred to 
as "PlayPLatPlayltemO". 

A PlayPL function using the Chapter to specify a reproduction start position in the PL time axis is referred to as "Play- 
PLatChapter()". 

45 a PlayPL function using the time information to specify a reproduction start position in the PL time axis is referred to as 
"PlayPLatSpecifiedTimeQ". 

The writing format of the navigation command in the Movie Object resembles the writing format of the navigation command 
in DVD. This makes it possible to efficiently transfer the disc contents from DVD to BD-ROM. The below-identified 
international publication, as a prior art, discloses the Movie Object. Please refer to the international publication for detail 
50 of the Movie Object. 

International Publication: WO 2004/074976 

Up to now, the Movie Object has been described. The following describes BD-J Object. 
<BD — J Object> 

BD-J Object is a dynamic scenario of the BC-J mode that is written in the Java ™ programming environment, and is 
55 stored in a file "00001 .bobj". The BD-J Object differs from the Movie Object in that it does not contain a command directly 
written therein. That is to say, in the Movie Object, the control procedure is directly written in the navigation command. 
In contrast, in the BD-J Object, the control procedure is indirectly defined, with specifications for Java ™ applications 
being written in the application managementtable. Such an indirectdefinition enables the sharing of the control procedure, 
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in which a control procedure is shared by a plurality of dynamic scenarios, to be performed efficiently. 
[0050] The reproduction of PlayList using Movie Object is achieved by writing a navigation command (PlayPL com- 
mand) that instructs to reproduce a PlayList. The reproduction of PlayList using BD-J Object is achieved by incorporating 
the PlayList management table, which indicates PlayList reproduction procedures, into the BD-J Object. 
Here, the Java ™ applications in the BD-J mode will be described. It is presumed here that fully mounted in the Java 
™ platform in the BD-J mode are Java2Micro_Edition (J2ME) Personal Basis Profile (PBP 1 .0), and Globally Executable 
MHP specification (GEM1 .0.2) for package media targets. 

[0051] The Java ™ applications in the BD-J mode are controlled by the Application Manager via the xlet interface. 
The xlet interface has four states : "loaded" , "paused" , "active" , and "destroyed" 

The above-described Java ™ platform includes a standard Java™ library for displaying image data conforming to JFIF 
(JPEG), PNG, or the like. With this construction, the Java ™ applications can achieve a GUI framework that is different 
from the GUI achieved by the IG stream in the HDMV mode. The GUI framework in the Java ™ applications includes 
the HAVi framework defined in GEM1 .0.2 and the remote control navigation mechanism in GEM1 .0.2. 
[0052] With such a construction, the Java™ applications can achieve a screen display in which a moving image is 
displayed on the screen together with buttons, texts, and online display (contents of BBS), and it is possible to perform 
operations on the displayed screen. 

The substance of the Java™ applications is the Java™ archive file (00001 .jar) stored in the BDJA directory under the 
BDMV directory shown in Fig. 2. 

[0053] The below-identified international publications, as prior arts, discloses the BD-J Object. Please refer to the 
international publications for detail of the BD-J Object. 

International Publications: WO 2004/045840 A1 

WO 2005/036555 A1 
WO 2005/036546 A1 

Up to now, the BD-J Object has been described. 
<sound.bdmv> 

The sound. bdmv will be described. The sound. bdmv is a file for storing audio data that is output as a click sound when 
a menu drawn in the IG stream or the GUI framework of a Java ™ application is operated (such audio data is referred 
to as sound data). 

[0054] Fig. 1 1 shows the structure of the file sound. bdmv. The file sound. bdmv includes Sound Data() and Sound 
lndex(). The Sound Data() includes a plurality of pieces of sound data (sound_data (0), sound_data (1 ) ...). Among these 
pieces of sound data, sound_data (0) is a sound source of the 1 st click sound that is output when the menu is operated, 
and sound_data (1) is a sound source of the 2 nd click sound that is output when the menu is operated. These sound 
data are identified by identifiers called sound IDs. 

[0055] The Sound lndex() includes: the number of sounds (number_of_sound_entries), an index to sound_data(0), 
an index to sound_data(1 ) ... 

Each index includes: sound_attributes that indicates an attribute of the sound such as monaural or stereo; sound_data_ 
start_address that indicates the address of a corresponding piece of sound data; and sound_data_length that indicates 
the sequential length of the corresponding piece of sound data. 

[0056] As shown in Figs. 2-6, the sources of the sounds used in the motion picture are multiplexed in the AVClip as 
the Primary audio stream. This arrangement is made for the purpose of supplying the Primary audio stream, which 
provides sound/voice in the motion picture, at the same time when the video stream is read out. On the other hand, the 
file sound. bdmv, in which the click sound for a menu Operation by the user is stored, is recorded on the BD-ROM 
separately from the AVClip. Since the file sound. bdmv is recorded separately from the AVClip, when sound data is tried 
to be output during a reading out of the AVClip, the optical pickup jumps to read the file sound. bdmv. When this happens, 
the reading out of the AVClip is interrupted and the reproduction of the AVClip is interrupted. 

[0057] To prevent such interruption of the AVClip reproduction, it is necessary to preload the file sound. bdmv in the 
buffer when the AVClip is not reproduced. That is to say, it is necessary to preload sound data of the file sound. bdmv 
prior to the AVClip reproduction. This completes the description of the file sound. bdmv. 
<lndex.bdmv> 

The Index. bdmv is a table that indicates the Movie Object or the BD-J Object that constitutes a title. 

[0058] The Index. bdmv defines the Movie Object or the BD-J Object that is a component of a Title. 

The below-identified international publication discloses the Index. bdmv. Please refer to the international publication for 

detail of the Index. bdmv. 

International Publication: WO 2004/025651 A1 

Up to now, the BD-ROM 100 has been described. 
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[0059] <Local Storage 200> 

The following describes the local storage 200 that is a recording medium of the present invention. Fig. 12 shows an 
internal structure of the local storage 200. As shown in Fig. 12, the recording medium of the present invention can be 
produced by improving the application layer. 

5 The fourth row of Fig. 12 shows the local storage 200. The third row shows tracks on the local storage 200. Fig. 12 
shows the tracks in the state where they are horizontally extended though they are in reality formed spirally in order 
from the inner circumference to the outer circumference of the local storage 200. The tracks include a lead-in area, a 
volume area, and a lead-out area. The volume area of Fig. 12 has a layer model that includes a physical layer, a file 
system layer, and an application layer. The 1 st row of Fig. 12 shows an application layer format (application format) of 

10 the local storage 200 represented by a directory structure. 

[0060] In the directory structure shown in Fig. 12, there is a subdirectory "organization^ " under a root directory. Also, 
there is a subdirectory "disc#1" under the directory "organization^ ". The directory "organization^ " is assigned to a 
specific provider of a motion picture. The directory "disc#1" is assigned to each BD-ROM provided from the provider. 
[0061 ] With this construction in which the directory assigned to a specific provider includes directories that corresponds 

*5 to BD-ROMs, download data for each BD-ROM is stored separately. Similarly to the information stored in the BD-ROM, 
under the subdirectory "disc#1 ", the following information is stored: PlayList information ("00002. mpls"); Clip information 
( M 00002.clpi M ); AVClip ("00001 .m2ts"); BD-J Object ("00002. bobj"); Java™ archive file ("00002.jar"); click sound data 
("sound. bdmv"); and Movie Object ("MovieObject.bdmv"). 

[0062] The following describes the PlayList information. Clip information, andAVClip that are the components of the 
20 local storage 200. 

<Local Storage 200 Structure 1 : AVCIip> 

The AVClip ("00002. m2ts") in the local storage 200 constitutes a SubClip. The SubClip is an AVClip that includes an 
elementary stream that is decoded and reproduced simultaneously with the MainClip. The SubClip is classified into 
"Primary audio stream", "Secondary audio stream", "Presentation Graphics (PG) stream", and "Interactive Graphics (IG) 
25 stream" (hereinafter, a SubClip may also be referred to as an Out-of-MUX stream). 

In the present embodiment, it is presumed that "00002. m2ts" shown in Fig. 1 2 is a SubClip in which the Secondary audio 
stream, PG stream, and IG stream are multiplexed. The Secondary audio stream will be described in detail from now on. 
<Out-of-MUX Stream Explanation 1 : Secondary Audio Stream> 

While the Primary audio stream is an audio stream that includes what is called main sound/voice, the Secondary audio 

30 stream is an audio stream that includes what is called sub-sound/voice. When the SubClip is reproduced, the sound/ 
voice for reproduction is output after the sound/voice of the Primary audio stream is mixed with the sound/voice of the 
Secondary audio stream. The sound/voice to be included in the Secondary audio stream includes, for example, "com- 
mentary sound/voice". When the main sound/voice included in the Primary audio stream is the sound/voice of the motion 
picture, and the sub-sound/voice included in the Secondary audio stream is the commentary sound/voice of the director, 

35 the sound/voice of the motion picture is output after it is mixed with the commentary sound/voice. 

[0063] The Secondary audio stream is recorded on only the local storage 200 for reproduction, but is not recorded 
on the BD-ROM. The Primary audio stream may be recorded on the BD-ROM and/orthe local storage 200. The encoding 
codec for the Primary audio stream may be different from the encoding codec for the Secondary audio stream. 
[0064] Fig. 13 shows elementary streams that are multiplexed into the SubClip. In Fig. 13, PG streams having PIDs 

^0 "0x1 200" to "0x1 21 F", IG streams having PIDs "0x1 400" to "0x1 41 F", and Secondary audio streams having PIDs "0x1 A00" 
to "0x1 A1 F" are multiplexed. Here, the PIDs of the PG streams and the IG streams in the SubClip are the same as the 
PIDs of the PG streams and the IG streams in the MainClip, but any of the PIDs of the 32 Secondary audio streams is 
not the same as any of the PIDs of the 32 Primary audio streams since they differ in the upper byte. 
[0065] Fig. 14 shows PID assignment map in the BD-ROM standard. According to Fig. 14, "0x0100" is assigned to 

45 "program_map", "0x1001" is assigned to "PCR", "0x1011" is assigned to the video stream, the zone from "0x1100" to 
"0x1 1 1 F" is assigned to the Primary audio streams, the zone from "0x1200" to "0x121 F" is assigned to the PG streams, 
the zone from "0x1 400" to "0x1 41 F" is assigned to the IG streams, and the zone from "0x1 A00" to "0x1 A1 F" is assigned 
to the Secondary audio streams. As understood from the PID assignment map, the zone assigned to the Primary audio 
streams is different from the zone assigned to the Secondary audio streams. 

50 [0066] Fig. 15A shows the internal structure of the Secondary audio stream. 

As shown in Fig. 1 5A, the Secondary audio stream is composed of a plurality of audio frames. Fig. 1 5B shows an example 
of the audio frame. The audio frame of the Secondary audio stream includes metadata. 

Fig. 15C shows an internal structure of the metadata. As shown in Fig. 15C, the metadata is composed of "downmixing 
information" and "gain control information". 
55 [0067] The downmixing information is information for downmixing. The downmixing is a conversion in which the number 
of audio reproduction channels is adjusted to be smaller than the number of encoding channels. The downmixing 
information defines a conversion coefficient matrix for the downmixing so that the reproduction apparatus can perform 
the downmixing. For example, the downmixing enables an audio stream in 5.1 channels to be reproduced in 2 channels. 
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[0068] The gain control information is used to increase or decrease the gain of the audio output of the Primary audio 
stream. In the present embodiment, the gain control information is only used to decrease the gain. Fig. 15D shows an 
example of the gain control information. As understood from this, the metadata of the Secondary audio stream can 
decrease in real time the output of the Primary audio stream that is reproduced simultaneously with the Secondary audio 
stream. When combining (mixing) the Primary and Secondary audio streams, it is not necessary to control in real time 
the gains of the two audio streams. Since the Primary and Secondary audio streams in a pair are preliminarily known, 
it is only necessary in mixing (combining) them to decrease the gain of the Primary audio stream, without changing the 
gain of the Secondary audio stream. 

[0069] It is possible to store only valid gain control information for a duration starting from the time specified by "mark_ 
time_stamp" of PlayListMark. 

Fig. 1 6 shows how the sound volume of the Primary audio stream is controlled by the metadata in the Secondary audio 
stream. In Fig. 16, the 1 st row shows the time axis, the 2 nd row shows the reproduction output of the Primary audio 
stream that can be mixed, the 3 rd row shows the reproduction output of the Secondary audio stream, and the 4 th row 
shows metadata multiplexed in the Secondary audio stream. 

[0070] The metadata disposed in an audio frame that corresponds to reproduction time t1 suppresses the sound 
volume of the reproduction output of the Primary audio stream as a whole. The metadata disposed in an audio frame 
that corresponds to reproduction time t2 restores the sound volume of the reproduction output of the Primary audio 
stream. With this arrangement in which metadata is disposed at reproduction times t1 and t2, it is possible to avoid a 
situation in which the sound volume of the reproduction output of the Primary audio stream is combined with that of the 
Secondary audio stream to cause the speaker to be broken. 

When the gain adjustment for mixing is performed in real time using the gain control information of the Secondary audio 
stream, it is only necessary for the gain control information stored in each audio frame of the Secondary audio stream 
to specify that the gain of the Primary audio stream is decreased by a predetermined amount. This method, which always 
secures an appropriate gain control, is more convenient than, for example, a case where the mixing reproduction is 
performed by jumping into the period between the reproduction times t1 and t2 using a special reproduction. 
[0071] <Local Storage 200 Structure 2: PlayList lnformation> 

Next, the PlayList information in the local storage 200 will be described. A file (00002. mpls) to which extension "mpls" 
is attached is information that defines PlayList (PL) by combining two types of presentation paths called Main Path and 
Subpath. Fig. 17 shows the data structure of the PlayList information. As shown in Fig. 17, the PlayList information 
includes: MainPath information (MainPathQ) that defines MainPath; PlayListMark information (PlayListMark()) that de- 
fines chapter; and Subpath information (Subpath()) that defines Subpath. The internal structures of the PlayList infor- 
mation and the Play Item information are the same as those of the BD-ROM, and the description thereof is omitted here. 
<PlayList Information Explanation 1: Subpath lnformation> 

While the MainPath is a presentation path that is defined in terms of the MainClip that is the main image, Subpath is a 
presentation path that is defined in terms of the SubClip that is synchronized with the MainPath. 
[0072] Fig. 18 shows the internal structure of the Subpath information with close-ups. As indicated by the arrow hcO, 
each Subpath includes: SubPath_type that indicates the type of SubClip; and one or more pieces of SubPlayltem 
information (■■■ SubPlayltem()—). 

The lead line hc1 indicates the close-up of the structure of the SubPlayltem information. As the lead line hc1 indicates, 
the SubPlayltem information includes: "Clip_information_file_name", "Clip_ocdec_identifier", "ref_to_STC_id[0]", 
"SubPlayltem_ln_time", "SubPlayltem_Out_time", "sync_Playltem_id", and "sync_start_PTS_of_Playltem". 
[0073] The "Clip_information_file_name" is information that uniquely specifies a SubClip that corresponds to Sub- 
Playltem, by writing the file name of the Clip information. 
The "Clip_codec_identifier" indicates an encoding method of the AVCIip. 
The "ref_to_STC_id[0]" uniquely indicates a STC_Sequence that is the target of the Playltem. 

The "SubPlayltem_ln_time" is information that indicates the start point of SubPlayltem in the reproduction time axis of 
the SubClip. 

[0074] The "SubPlayltem_Out_time" is information that indicates the end point of SubPlayltem in the reproduction 
time axis of the SubClip. 

The "sync_Playltem_id" is information that uniquely specifies, among Playltems constituting the MainPath, Playltem 
with which the SubPlayltem is to be synchronized. The "SubPlayltem_ln_time" is present in the reproduction time axis 
of the Playltem specified by the "sync_Playltem_id". 

[0075] The "sync_start_PTS_of_Playltem" indicates the position of the start point of the SubPlayltem specified by the 
SubPlayltem_ln_time, in the reproduction time axis of the Playltem specified by the "sync_Playltem_id". 
<Subpath Information Detail 1: SubPath type> 

Up to now, the SubPath information has been described. From now on, the SubPath_type will be explained. The SubPath_ 
type is set to be a value among 0 to 255 to indicate the type of the SubPath that is defined by the SubPath information. 
[0076] When the SubPath_type is set to 5, it indicates that the SubPath defined by the SubPath information is the 
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Primary audio presentation path. The Primary audio presentation path is used when the audio stream, which is to be 
replaced with the Primary audio stream referred to by the MainPath (Playltem) before it is reproduced, is stored in the 
SubPath (SubPlayltem). 

When the SubPath_type is set to 6, it indicates that the current SubPath information defines the Presentation Graphics 
5 presentation path for addition or replacement. Here, the "addition or replacement" concerns the addition of a PG stream 
to the PG streams that can be reproduced by the Playltem information, or the replacement of a PG stream with any of 
the PG streams that can be reproduced by the Playltem information. 

[0077] When the SubPath_type is set to 7, it indicates that the current SubPath information defines the Interactive 
Graphics presentation path for addition or replacement. Here, the "addition or replacement" concerns the addition of an 
10 |G stream to the IG streams that can be reproduced by the Playltem information, or the replacement of an IG stream 
with any of the IG streams that can be reproduced by the Playltem information. 

When the SubPath_type is set to 8, it indicates that the current SubPath information defines the Secondary audio 
presentation path for addition. Here, the "addition" concerns the addition of a Secondary audio stream that is to be mixed 
with the reproduction sound/voice of the Primary audio stream that can be reproduced by the Playltem information. 

*5 For example, when a mixing reproduction of the Primary and Secondary audio streams is performed, two audio decoders 
and two mixers need to be operated and accordingly the reproduction type of the player needs to be acquiredpreliminarily, 
which is different from the case where only the ordinary Primary audio stream is reproduced. For this purpose, Subpath_ 
type or the PID of the STN_table is used to notify the player, before a reproduction, that there is a Secondary audio 
stream that is to be reproduced in synchronization therewith. 

20 [0078] Up to now, the Subpath_type has been described. 

<Subpath Information Detail 2: Relationships among Three Parties> 

It should be noted that the "three parties" mentioned here are SubClip in the local storage 200, PlayList information in 
the local storage 200, and MainClip in the BD-ROM. 

Fig. 19 shows the relationships among SubClip in the local storage 200, PlayList information in the local storage 200, 
25 and MainClip in the BD-ROM. In Fig. 19, the 1 st row shows SubClip in the local storage 200. As shown in the 1 st row, 
the SubClip in the local storage 200 is any of the Secondary audio stream, PG stream, and IG stream. Any of these 
streams is subjected to the synchronous reproduction as a Subpath. 

[0079] The 2 nd row of Fig. 19 shows two time axes that are defined by the PlayList information. Of these, the lower 
time axis is the Playltem time axis defined by the Playltem information, and the upper time axis is the SubPlayltem time 
30 axis defined by the SubPlayltem information. 

As shown in Fig. 19, "SubPlayltem_Clip_information _file_name" of the SubPlayltem information indicates one among 
M .m2ts" files stored in the STREAM directory to specify a reproduction section for a SubClip. 

[0080] The "SubPlayltem_ln_time" and "SubPlayltem_Out_time" play a role of defining the start point and the end 
point of the reproduction section. 
35 The arrow "Sync_Playltem_id" plays a role of specifying synchronization, namely, specifying a Playltem to be synchro- 
nized with. The arrow "sync_start_PTS_of_P lay Item" plays a role of determining the position of "SubPlayltem_ln_time" 
in the Playltem time axis. 

[0081] Fig. 20 shows the EP_map and the Playltem time axis that are set for the MainClip and the EP_map and the 
SubPlayltem time axis that are set for the SubClip. 
^o in Fig. 20, the middle row and the lower 4 th to lower 1 st rows show the Playltem time axis, picture sequence, MainClip 
time axis, EP_map, and TS packet sequence shown in Fig. 10. 

Also, the upper 1 st to upper 3 rd rows show the TS packet sequence, EP_map, and SubClip time axis, and the upper 4 th 
row shows the Subltem time axis. 

[0082] Up to now, the SubPath information has been described. <STN_table> 
45 What is unique to the PlayList information in the local storage 200 is the STN_Table. The following describes the PlayList 
information in the local storage 200. 

The STN_table is a table that indicates reproducible streams among a plurality of elementary streams multiplexed in 
the AVCIips specified by the Clip Information file name in the Playltem information, and among Out_of_MUX streams 
specified by the Clip_lnformation_file_name in the information. More specifically, the STN_table is generated by asso- 
50 ciating the Stream_attributes with the Stream entries respectively corresponding to the plurality of elementary streams 
multiplexed in the MainClips and to the Out_of_MUX streams multiplexed in the SubClips. 

[0083] Fig. 21 shows the internal structure of the STN_table. As shown in Fig. 21 , the STN_table includes a plurality 
of pairs (entry-attribute) of an entry of the STN_table and an attribute, and shows the number of the pairs (number_of_ 
video_stream_entries, number_of_audio_stream_entries, number_of_PG_stream_entries, number_of_IG_stream_en- 
55 tries). 

[0084] In the STN_table, as indicated by the parenthesis sign "{", each entry-attribute pair correspond to any of a video 
stream, Primary audio stream, Secondary audio stream, PG stream, and IG streams that are permitted to be reproduced 
by the Playltem. In particular, it should be noted here that pairs of a stream entry and a Stream_attribute in the Secondary 



14 



EP 1 713 075 A1 



audio stream are associated with Comb_info_Secondary_audio_Primary_audio. 
The entry — attribute will now be described in detail. 

[0085] Fig. 22A shows Stream_attribute corresponding to a video stream. 

The Stream attribute for video stream includes; "Video_format" that indicates a video stream display method; "frame_ 
rate" that indicates a display frequency of the video stream; and so on. 

Fig. 22B shows Stream_attributes corresponding to the Primary audio stream and the Secondary audio stream. 
[0086] The Stream_attributes for the Primary and Secondary audio streams includes: "stream_coding_type" that 
indicates the audio stream encoding method; "audio_presentation_type" that indicates the channel construction of the 
corresponding audio stream; "Sampling_frequency" that indicates the sampling frequency of the corresponding audio 
stream; and "audiojanguage code" that indicates the language attribute of the audio stream. 

[0087] Fig. 22C shows Stream_entry for the video streams. As shown in Fig. 22C, the Stream_entry for the video 
stream includes "ref_to_Stream_PID_of_Main_Clip" that indicates a PID used for demultiplexing the video stream. 
The Stream_attributes for the Primary audio stream, IG stream, and PG stream multiplexed in the MainClip have the 
format shown in Fig. 22C. 

[0088] Fig. 22D shows Sub_entries for the streams (in the following description, referred to as Secondary audio 
streams) multiplexed in the SubClip. The Stream_entry for the Secondary audio stream includes "ref_to_Sub_Path_id" 
that indicates SubPath information that is referring to the Secondary audio stream; "ref_to_Sub_Clip_entry_id" that 
indicates the SubClip in which the Secondary audio stream is multiplexed; and "ref_to_stream_PID_of_Sub_Clip " that 
indicates the PID used for demultiplexing the Secondary audio stream. 

[0089] Fig. 22E shows the internal structure of Comb_info_Secondary_audio_Primary_audio that is associated with 
a pair of Stream_entry and Stream_attribute for the Secondary audio stream. The Comb_info_Secondary_audio_ 
Primary_audio is composed of: number_of _primary_audio_stream_ref_entries that indicates, for each Secondary audio 
stream, the total number of Primary audio streams that can be combined with the Secondary audio streams; and Primary_ 
audio_stream_id_ref[0] ... [n] that indicate the stream numbers of the Primary audio streams that can be combined during 
reproduction. 

[0090] Fig. 23 shows the specification of the Primary audio streams by Comb_info_Secondary_audio_Primary_audio. 
In Fig. 23, the right-hand side shows 32 Secondary audio streams, and the left-hand side shows 32 Primary audio 
streams. The arrow ym1 indicates specification by Comb info_Secondary_audio_Primary_audio of Secondary audio 
stream#1. In this way, the Comb_info_Secondary_audio_Primary_audio that is set for each Secondary audio stream 
specifies uniquely one or more Primary audio streams with which the reproduction output of the Secondary audio stream 
can be mixed. Such a construction makes it possible to set, at the time of authoring, whether or not to mix the Secondary 
audio stream, depending on the audio attribute. For example, it is possible to set not to mix the Secondary audio stream 
with, during the reproduction, Primary audio streams that have a predetermined attribute, and to set to mix the Secondary 
audio stream with, during the reproduction. Primary audio streams that have attributes other than the predetermined 
attribute. 

[0091 ] Up to now, the PlayList information in the local storage 200 has been explained. This completes the explanation 
of the local storage 200. 
<Virtual Package> 

From now on, the virtual package will be described. Fig. 24 shows a virtual package that is generated by the reproduction 
apparatus 300. In Fig. 24, the upper left-hand side shows the recorded contents of the BD-ROM, and the lower left-hand 
side shows the recorded contents of the local storage 200. Also, the right-hand side shows the structure of the virtual 
package. 

[0092] The reproduction apparatus obtains a virtual file system by combining the AVCIips, Clip information, and PlayList 
information stored in the BD-ROM and the local storage 200. 
More specifically, for this combination, the reproduction apparatus: 

i) virtually adds Playl_ist(00002.mpls) in the Local Storage to the MPLS directory in the BD-ROM; 

ii) virtually adds Clip information (00002. dpi) in the Local Storage to the CLPI directory in the BD-ROM; and 

iii) virtually adds AVCIip#2 (00002. m2ts) in the Local Storage to the STREAM directory in the BD-ROM. 

[0093] With the above-described combination, the virtual package shown in the right-hand side of Fig. 24 is generated. 
Up to now, the recording medium has been described. From now on, the reproduction apparatus will be described. 
Fig. 25 shows the internal structure of the reproduction apparatus of the present invention. The reproduction apparatus 
of the present invention is produced industrially based on the internal structure shown in Fig. 25. The reproduction 
apparatus of the present invention is mainly composed of two parts: a system LSI; and a drive apparatus, and can be 
produced industrially by mounting these parts in the cabinet and on the substrate. The system LSI is an integrated circuit 
in which various processing units, which perform functions of the reproduction apparatus, are included. The reproduction 
apparatus produced in this way includes: a BD-ROM drive 1a, a bus 1b, read buffers 2a and 2b, demultiplexers 3a and 
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3b, a video decoder 4, a video plane 5, buffers 6a, 6b and 6c, audio decoders 7a and 7b, a DownMix/DownSample 8, 
mixers 9a and 9b, a switch 1 0a, an encoder 1 0b, an Interactive Graphics decoder 11 , an Interactive Graphics plane 1 2, 
a Presentation Graphics decoder 13, a Presentation Graphics plane 14, a JPEG decoder 15, a Still plane 1 6, a combining 
unit 1 7, STC generating units 1 8a and 1 8b, ATC generating units 1 9a and 1 9b, a memory 21 , a controller 22, a PSR 
5 set 23, a PID conversion unit 24, a communication unit 25, an operation receiving unit 26, and an HDMI transmission/ 
reception unit 27. 

[0094] The BD-ROM drive 1 a performs the loading/ejection of the BD-ROM, and performs accesses to the BD-ROM. 
The bus 1 b transfers thereon the TS packets read from the BD-ROM, and the TS packets read from the local storage 200. 
The read buffer 2a is a FIFO memory in which the TS packets read from the BD-ROM 100 or the local storage 200 are 
10 stored by the First-In-First-Out method. 

[0095] The read buffer 2b is a FIFO memory in which the TS packets read from the local storage 200 are stored by 
the First-In-First-Out method. 

The demultiplexer 3a outputs TS packets having the PIDs that are notified from the PID conversion unit 24, among the 
TS packets that are transferred on the bus and have PIDs 0x101 1 , 0x1 100-0x1 1 1 F, 0x1200-0x121 F, and 0x1400-141 F, 

is to the video decoder 4, the audio decoder 7a, the IG decoder 1 1 , and the PG decoder 13. 

[0096] The demultiplexer 3b demultiplexes TS packets having PIDs 0x1 A00-0x1 A1 F, namely TS packets constituting 
the Secondary audio stream, among the TS packets that are transferred on the bus 1b. The demultiplexing by the 
demultiplexer 3b is achieved by comparing the reference values of the PIDs written in the stream_entries which, among 
the stream entries of the Secondary audio streams containing the STN_table, correspond to the stream numbers stored 

20 in the PSR14, with the PIDs of the TS packets transferred on the bus 1b, and outputting the TS packets, which have 
PIDs that are found to match in the comparison, to the audio decoder 7b via the buffer 6b. It should be noted here that 
if there is only one reproducible Secondary audio stream, it is only necessary to compare the upper byte "1 A" of the PID 
reference value written in the stream_entry, with the upper byte "1 A" of the PI D of the TS packet transferred on the bus 
1 b. This is because since there is no other Secondary audio stream, it is only necessary to refer to the upper byte of the 

25 pid that indicates that it is the Secondary audio stream. 

[0097] If there are a plurality of reproducible Secondary audio streams, it is necessary to compare the lower byte (a 
value ranging from 0x00 to 0x1 F) of the PID reference value written in the stream_entry, with the lower byte (a value 
ranging from 0x00 to 0x1 F) of the PID of the TS packet transferred on the bus 1b, in addition to comparing the upper 
byte "1 A" of the PID reference value written in the stream_entry, with the upper byte "1 A" of the PID of the TS packet 

30 transferred on the bus 1 b. This is because since there are a plurality of Secondary audio streams, it is necessary to refer 
to the lower byte of the PID so as to uniquely identify the Secondary audio stream. 

[0098] Here, since the TS packets read from the BD-ROM and the local storage 200 are transferred on the bus 1b, 
the demultiplexer 3a and the demultiplexer 3b can deal with these TS packets read from the BD-ROM and the local 
storage 200 as one transport stream, and store them into the buffer as one transport stream. The PIDs assigned to the 

35 TS packets constituting the Primary audio stream are present in a separate zone in the PID assignment map from a 
zone in which the PIDs assigned to the TS packets constituting the Secondary audio stream are present. With this 
construction, while inputting these TS packets as one transport stream, the demultiplexer 3a and the demultiplexer 3b 
can output the Primary audio stream and the Secondary audio stream as different elementary streams. Here, the de- 
multiplexer 3a and the demultiplexer 3b can supply the plurality of audio streams multiplexed in one transport stream to 

40 the decoder, by performing the same procedure as in demultiplexing them. It is accordingly possible for the demultiplexer 
3a and the demultiplexer 3b to supply the Primary audio stream and the Secondary audio stream to the respective 
decoders in a manner compatible with a demultiplexer that demultiplexes only such TS packets that have a predetermined 
PID, from one transport stream. 

[0099] There might be a case where only one demultiplexer is provided for the sake of mounting. In such a case also, 
45 assigning different PIDs to the Primary audio stream and the Secondary audio stream is effective. 

Up to now, the BD-ROM drive 1 a, the bus 1 b, the read buffers 2a and 2b, and the demultiplexers 3a and 3b have been 
described. 

The video decoder 4 decodes a plurality of PES packets output from the demultiplexer 3a to obtain non-compressed 
pictures, and writes the obtained pictures to the video plane 5. 
50 [0100] The video plane 5 is a plane for storing non-compressed pictures. The "plane" is a memory area in the repro- 
duction apparatus for storing pixel data of one screen. The resolution of the video plane 5 is 1 920 x 1 080. The picture 
data stored in the video plane 5 is composed of a plurality of pieces of pixel data each of which is represented by a 1 6- 
bit YUV value. 

[0101] The buffer 6a stores therein, among the TS packets output from the demultiplexer 3a and having PIDs in a 
55 range from 0x1 100 t 0x1 1 1 F, TS packets having PIDs of the audio streams to be reproduced, by the First-In-First-Out 
method, and supplies the stored TS packets to the audio decoder 7a. 

The buffer 6b stores therein, among the TS packets output from the demultiplexer 3b and having PIDs in a range from 
0x1 A00 1 0x1 A1 F, TS packets having PIDs of the audio streams to be reproduced, by the First-In-First-Out method, and 
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supplies the stored TS packets to the audio decoder 7b. 

[0102] The buffer 6c is a memory for preloading therein the file sound. bdmv that is read out from the BD-ROM/local 
storage. It is preferable that the preloading into the buffer 6c is performed when the BD-ROM is loaded or when titles 
are changed. This is because when the file sound. bdmv is tried to be read out during a reproduction of an AVClip, the 
5 optical pickup performs a seek to read out a file that is different from the AVClip. On the other hand, when the BD-ROM 
is loaded or when titles are changed, an AVClip is hardly being reproduced. Therefore, if the file sound. bdmv is always 
read out when the BD-ROM is loaded or when titles are changed, the response of the apparatus is increased and the 
AVClip reproduction is less apt to be interrupted. 

[0103] The audio decoder 7a converts the TS packets stored in the buffer 6a into PES packets, decodes the PES 
10 packets to obtain non-compressed audio data in the LPCM state, and outputs the obtained audio data. This achieves 
a digital output for the Primary audio stream. 

The audio decoder 7b converts the TS packets stored in the buffer 6b into PES packets, decodes the PES packets to 
obtain non-compressed audio data in the LPCM state, and outputs the obtained audio data. This achieves a digital 
output for the Secondary audio stream. 

15 [0104] The DownMix/DownSample 8 converts the digital audio output from the audio decoder 7a to match the digital 
audio output from the audio decoder 7b in the audio attributes. Here, the audio attributes are the sampling frequency 
and/or the number of channels. They are made to be compatible with each other. The DownMix/DownSample 8 or the 
mixer 9a, in accordance with the gain control information extracted by the audio decoder 7b, also performs the process 
of reducing the gain on the side of the Primary audio stream in accordance with the metadata multiplexed in the Secondary 

20 audio stream. 

The mixer 9a performs a mixing of the digital audio in the LPCM state output from the audio decoder 7a with the digital 
audio in the LPCM state output from the audio decoder 7b. 

[0105] The mixer 9b performs a mixing of the digital audio in the LPCM state output from the mixer 9a with the sound 
data stored in the buffer 6c. The mixing by the mixer 9b is performed when the CPU 22 decodes a navigation command 

25 that is intended to output a click sound, or a byte code that is intended to output a click sound. 

The switch 10a switches between (a) supplying, to the audio decoders 7a and 7b, the TS packets of the Primary audio 
stream demultiplexed by the demultiplexer 3a and the TS packets of the Secondary audio stream demultiplexed by the 
demultiplexer 3b, and (b) not supplying to the audio decoders 7a and 7b but outputting only the elementary streams to 
other devices. This operation of not supplying the TS packets of the Primary audio stream to the audio decoders 7a and 

30 7b but outputting only the elementary streams to other devices is called "pass-through output". A conversion unit (not 
illustrated) that converts the TS packets into the elementary streams (that removes the TS/PES headers) is embedded 
in the switch 10a. 

The encoder 1 0b, when the audio data in the LPCM state, which is obtained as a result of decoding by the audio decoder 
7a and the audio decoder 7b and mixing by the mixer 9b, is sent as a surround sound/voice via a digital interface such 
35 as S/PDIF, encodes and compresses the mixed LPCM into the Dolby Digital (DD) format or the Dolby Theater System 
(DTS) format. 

[0106] The Interactive Graphics (IG) decoder 1 1 decodes an IG stream read out from the BD-ROM 100 or the local 
storage 200, and writes the obtained non-compressed graphics into the IG plane 12. 

The Interactive Graphics (IG) plane 12 is a plane into which the non-compressed graphics obtained by the IG decoder 
40 1 1 are written. Also, characters or graphics drawn by applications are written into the IG plane 12. 

[0107] The Presentation Graphics (PG) decoder 13 decodes an PG stream read out from the BD-ROM 100 or the 
local storage 200, and writes the obtained non-compressed graphics into the PG plane 14. 

The Presentation Graphics (PG) plane 14 is a memory having an area of one screen, and stores one screen of non- 
compressed graphics. 

45 [0108] The JPEG decoder 15 decodes JPEG data recorded in the BD-ROM 100 or the local storage 200, and writes 
the obtained data into the Still plane 16. 

The Still plane 1 6 is a plane into which the non-compressed graphics obtained by decoding the JPEG data are written. 
Also, characters or graphics drawn by applications are written into the IG plane 12. The graphics data is used as the 
"wallpaper" of the GUI frame work written by a Java ™ application. 
50 [0109] The combining unit 17 combines the data stored in the IG plane 12, the data stored in the PG plane 14, the 
data stored in the video plane 5, and the data stored in the Still plane 16 to obtain a complex image. 
The STC generating units 18a and 18b generate a System Time Clock (STC) in accordance with an instruction by the 
controller 22, and adjust the operation timing of each decoder. 

[0110] The ATC generating units 19a and 19b generate Arrival Time Clock (ATC) in accordance with an instruction 
55 by the controller 22, and adjust the operation timing of the demultiplexer. The memory 21 is a memory for storing current 
PL information and current Clip information. The current PL information is PL information that is currentlyprocessed, 
among a plurality of pieces of PlayList information stored in the BD-ROM. The current Clip information is Clip information 
that is currently processed, among a plurality of pieces of Clip information stored in the BD-ROM/local storage. 
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[01 1 1] The controller 22 achieves a reproduction control of the BD-ROM by decoding a Java ™ application referred 
to by a Movie Object or a BD-J Object stored in the MovieObject.bdmv, and performing a PlayList reproduction in 
accordance with the decoding result (that is to say, a reproduction control in accordance with the current PL information). 
The controller 22 also performs the above-mentioned ATS control and STC control. 

The PSR set 23 is a register embedded in the reproduction apparatus, and is composed of the 64 number of Player 
Setting/Status Registers (PSR) and the 4096 number of General Purpose Registers (GPR). Among the values (PSR) 
set in the Player Setting/Status Registers, PSR4 to PSR8 are used to represent the current reproduction point. 
[0112] The PID conversion unit 24 converts the stream numbers of the Primary audio streams and Secondary audio 
streams into PIDs based on the STN table, and notifies the PIDs as the conversion results to the demultiplexers 3a and 3b. 
The communication unit 25 achieves a communication function in the reading apparatus. When a URL is specified by 
a Java ™ application in the BD-Jmode, the communication unit 25 establishes a TCP connection or an FTP connection 
with a web site of the specified URL. The establishment of such a connection enables the Java ™ application to download 
data from the web site. 

[01 13] The operation receiving unit 26 receives specification of an operation from a user, and notifies User Operation 
information, which indicates the operation specified by the user, to the controller 22. 

The HDMI transmission/reception unit 27 receives, from an apparatus connected via HDMI (High Definition Multimedia 
Interface), information concerning the apparatus, and transmits the digital non-com pressed video, which is obtained as 
a result of decoding by the video decoder 4, to the apparatus via HDMI, together with LPCM audio data. 
[0114] Up to now, the hardware construction of the reproduction apparatus of the present invention has been explained. 
From now on, the software construction of the reproduction apparatus of the present invention will be explained. 
The controller 22 shown in Fig. 25 is shown in terms of the functions in Fig. 26. Fig. 26 shows the functions of the 
controller 22. As shown in Fig. 26, the controller 22 includes a PlayList processing unit 41 , a Procedure executing units 
42 and 43, a mixing control unit 44, and an ATC/STC control unit that causes the ATC generating units 19a and 19b 
and the STC generating units 1 8a and 1 8b to generate ATC and STC, respectively. 

[0115] These components perform the processes based on the PSR set 23. The following explains PSR1, PSR14, 
and PSR31. <PSR1> 

Fig. 27A shows the bit assignment in the PSR1 . 

As shown in Fig. 27A, the lower eight bits (b0-b7) among 32 bits of the PSR1 represents a stream number, and identifies 
one of a plurality of Primary audio streams whose entries are written in the STN table of the current Play Item. When 
the value set in the PSR1 changes, the reproduction apparatus reproduces a Primary audio stream corresponding to 
the set value after the change. The PSR1 is set to "OxFF" as the initial value, and then may be set to a value ranging 
from "1" to "32" by the reproduction apparatus. The value "OxFF" is an unspecified value and indicates that there is no 
Primary audio stream or that a Primary audio stream has not been selected. When the PSR1 is set to a value ranging 
from "1" to "32", the set value is interpreted as a stream number of a Primary audio stream. 
[0116] <PSR14> 

Fig. 27B shows the bit assignment in the PSR14. 

As shown in Fig. 27B, the lower eight bits (b0-b7) among 32 bits of the PSR1 4 represents a stream number, and identifies 
one of a plurality of Secondary audio streams whose entries are written in the STN_table of the current Play Item. When 
the value set in the PSR14 changes, the reproduction apparatus reproduces a Secondary audio stream corresponding 
to the set value after the change. The PSR1 4 is set to "OxFF" as the initial value, and then may be set to a value ranging 
from "1" to "32" by the reproduction apparatus. The value "OxFF" is an unspecified value and indicates that there is no 
Secondary audio stream or that a Secondary audio stream has not been selected. When the PSR14 is set to a value 
ranging from "1 " to "32", the set value is interpreted as a stream number of a Secondary audio stream. 
[0117] <PSR31> 

Fig. 27C shows the bit assignment in the PSR31 . 

As shown in Fig. 27C, bit 1 6 to bit 1 9 (b1 6-b1 9) among 32 bits of the PSR31 represents Player Profile information. When 
bit 16 to bit 19 are "0000"b, they indicate that the reproduction apparatus is shipped during a grace period, where ""b 
indicates bit values. Here, the grace period is a period during which reproduction apparatuses without predetermined 
functions may be shipped. The functions that can be omitted in the reproduction apparatuses shipped in the grace period 
include the audio mixing function. As a result, if bit 1 6 to bit 1 9 (b1 6-b1 9) of the PSR31 are set to "0000"b, namely if the 
Player Profile information is "0000"b, it indicates that predetermined functions including the audio mixing function are 
not mounted in the reproduction apparatus. 

[01 18] When the Player Profile information is "0001 "b, it indicates that the reproduction apparatus is shipped after the 
grace period. Since reproduction apparatuses shipped after the grace period are basically required to be mounted with 
all the functions to be provided, the Player Profile information being "0001 "b indicates that the reproduction apparatus 
is mounted with the mixing. 

[01 19] When the Player Profile information is "001 1 "b, it indicates that the reproduction apparatus is mounted with all 
the functions to be provided. Such a reproduction apparatus is mounted with all the functions regardless of whether it 
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is shipped before or after the grace period. As a result, when the Player Profile information is "001 1"b, it indicates that 
the reproduction apparatus is mounted with the sound mixing function. 

The PSR may include, as information indicating the mixing function, information that indicates the number of channels 
with which the reproduction apparatus can perform the mixing. 

5 [0120] Alternatively, the PSR may include information that indicates the number of audio output channels at the final 
output. For example, if an interface such as HDMI is connected for use, 5 . 1 -channel LPCM audio, which is the decoding 
result, can be output as LPCM. However, if the interface is, for example, S/PDIF, the audio is output as 2 channels (L/R), 
not as 5.1 channels, for which a compression by an encoder is required. Accordingly, if it is judged that the mixer is 
followed by an encoder connected to S/PDIF (for example, not connected to HDMI), the number of audio output channels 

10 at the final output can be set to 5.1 channels. If the mixer is not followed by an encoder, the number of audio output 
channels at the final output can be set to 2 channels after mixing. 
[0121] Up to now, the PSR set 23 has been described. 

From now on, the PlayList processing unit 41, the Procedure executing unit 42, and the Procedure executing unit 43 
will be described. 

15 <Functional Construction Detail 1 : PlayList Processing Unit> 

The PlayList processing unit 41 achieves the PL reproduction. The PlayList processing unit 41 reproduces portions of 
a video stream and a Primary audio stream that correspond to a section from ln_time to Out_time of the Playltem 
information, and to be in synchronization with this, the PlayList processing unit 41 causes the audio decoder 7b to 
reproduce a portion of a Secondary audio stream that corresponds to a section from Sub_Playltem_ln_time to Sub_ 

20 Playltem_Out_time of the SubPlayltem information. 

[0122] Fig. 28 is a flowchart showing the PlayList reproduction procedure by the PlayList processing unit. 
In this flowchart, first the current PL information (.mpls) is read out (step S101), and then steps S102 to S110 are 
performed. The steps S102 to S1 10 constitute a loop in which steps S103 to S1 10 are repeatedly performed for each 
piece of PI information that constitutes the current PL information until it is judged as Yes in step S109. In this loop 

25 process, the process-target Playltem is referred to as Playltem#x (Pl#x). The Playltem#x is initialized by being set to 
the starting Playltem of the current PL information (step S1 02). The condition for ending the loop is that the Playltem#x 
is the last Playltem of the current PL information (step S1 09). If it is judged in step S1 09 that Playltem#x is not the last 
Playltem of the current PL information, Playltem#x is set to the next Playltem in the current PL information (step S1 1 0). 
[0123] In the steps S103 to S1 10 that are repeatedly performed in the loop process, Clip information specified by Clip_ 

30 information_file_name of Playltem#x is read out onto the memory (step S103), ln_time of Playltem#x is converted into 
l-Picture address u using EP_map of the current Clip information (step S104), Out_time of Playltem#x is converted into 
l-Picture address v using EP_map of the current Clip information (step S1 05), l-Picture address next to l-Picture address 
v is obtained, and an address that precedes the obtained address by one is set to address w (step S107), and the BD- 
ROM 100 or the local storage 200 is instructed to read TS packets from l-Picture address u to address w (step S108). 

35 [0124] On the other hand, a video decoder or the like is instructed to output from mark time stamp of current PLMark 
to Out time of Playltem#x (step S106). In the steps S105 to S108, a portion of AVClip specified by Playltem#x is 
reproduced. 

After this, it is judged whether or not the Playltem#x is the last Playltem of the current PL information (step S109). 
[0125] If it is judged in step S1 09 that Playltem#x is not the last Playltem of the current PL information, Playltem#x is 
40 set to the next Playltem in the current PL information (step S1 1 0), and the control returns to step S1 03. As the above- 
described steps S103 to S1 10 are repeated, each piece of PI information that constitutes the current PL information is 
reproduced in sequence. 

<Functional Construction Detail 2: Procedure Executing Unit42> 

The Procedure executing unit 42, when necessity for changing a piece of Playltem information to another arises, or 
45 when the user instructed, through an operation, to change the streamnumber, executes a predetermined stream selection 
procedure, and writes a new stream number into PSR1. The reproduction apparatus reproduces the Primary audio 
stream according to the stream number written in PSR1. The setting of PSR1 thus enables a Primary audio stream to 
be selected. 

[0126] The reason why the stream selection procedure is executed when the Playltem information is changed is as 
50 follows. Since the STN_Table is provided in correspondence with each piece of Playltem information, it may happen 
that a Primary audio stream, which is reproducible according to a piece of Playltem information, cannot be reproduced 
according to another piece of Playltem information. 

The Procedure executing unit 42 brings PSR1 to change in status as shown in Fig. 29A. Fig. 29A shows the status 
transition of the value set in PSR1. It should be noted here that in Fig. 29A, the term "valid" means that the value of 
55 PSR1 is equal to or smaller than the number of entries written in the STN_table of the Play Item, and is decodable. 

[0127] The term "invalid" means that (a) the value of PSR1 is "0", (b) the value of PSR1 is larger than the number of 
entries written in the STN_table of the Play Item, or (c) decoding is not available even if the number of entries written in 
the STN_table of the Play Item is in the range from "1 " to "32". 
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The frames drawn with dotted lines in Fig. 29A indicate a procedure for determining the value of PSR when the status 
changes. The procedure for setting a value to PSR is classified into "Procedure when playback condition is changed" 
and "Procedure when Stream change is requested". 

[0128] The "Procedure when playback condition is changed" is a procedure to be executed when the reproduction 
5 apparatus has changed in the status due to some event that occurred to the apparatus. 

The "Procedure when Stream change is requested" is a procedure to be executed when the user requests some changing 
(in Fig. 29A, "Change a Stream"). 

The "Procedure when playback condition is changed" and "Procedure when Stream change is requested" are stream 
selection procedures and will be described in detail with reference to the flowcharts. 

10 [0129] The arrows in Fig. 29A symbolically indicate the status transition of PSR. 

The notes attached to the arrows indicate events that trigger each status transition. That is to say, Fig. 29A indicates 
that a status transition of PSR1 occurs if an event such as "Load Disc", "Change a Stream", "Start PlayList playback", 
"Cross a Playltem boundary", or "Terminate PlayList playback" occurs. It will be understood by referring to Fig. 29A that 
the above-mentioned procedures are not performed when a status transition of "Invalid Invalid" or "Valid^lnvalid" 

*5 occurs. On the other hand, status transitions of "Invalid^Valid" and "Valid^Valid" go through the dotted-line frames. 
That is to say, the above-mentioned "Procedure when playback condition is changed" or "Procedure when Stream 
change is requested" is performed when PSR1 is set to be "Valid". 
[0130] The following describes the events that trigger the status transition. 

The event "Load Disc" indicates that the BD-ROM has been loaded in the reproduction apparatus. In such loading, PSR1 
20 is once set to an unspecified value (OxFF). The event "Start PlayList playback" indicates that a reproduction process 
based on PlayList has been started. When such an event occurs, the "Procedure when playback condition is changed" 
is executed, and PSR1 is set to be "Valid". 

[0131] The event "Terminate PlayList playback" indicates that a reproduction process based on PlayList has been 
terminated. It is understood that when such an event occurs, the "Procedure when playback condition is changed" is 

25 not executed, and PSR1 is set to be "Invalid". 

The event "ChangeXXX" indicates that the user requested to change XXX (in Fig. 29A, Stream). If this event occurs 
when PSR1 is "Invalid" (indicated by "Cj1" in Fig. 29A), PSR1 is set to a value as requested. Even if the value that is 
set in this way indicates a valid stream number, the value set in PSR1 is treated as an "Invalid" value. That is to say, in 
the status transition triggered by the event "ChangeXXX", PSR never changes from Invalid to Valid. 

30 [0132] On the other hand, if event "Change a Stream" occurs when PSR1 is "Valid" (indicated by "cj2" in Fig. 29A), 
the "Procedure when Stream change is requested" is executed, and PSR1 is set to a new value. Here, the value set 
when the "Procedure when Stream change is requested" is executed may not be a value desired by the user. This is 
because the "Procedure when Stream change is requested" has a function to exclude an invalid value. If the event 
"Change stream" occurs when PSR1 is "Valid", PSR1 never changes from Valid to Invalid. This is because the "Procedure 

35 when Stream change is requested" assures that PSR1 does not become "Invalid". 

[0133] The event "Cross a Playltem boundary" indicates that a Play Item boundary has been passed. Here, the Play 
Item boundary is a boundary between two successive Play Items, namely a position between the end of a preceding 
Play Item and the start of a succeeding Play Item, among the two successive Play Items. If the event "Cross a Playltem 
boundary" occurs when PSR1 is "Valid", the "Procedure when playback condition is changed" is executed. After the 

40 execution of the "Procedure when playback condition is changed", the status of PSR1 either returns to "Valid" or transits 
to "Invalid". Since the STN_table is provided in correspondence with each Play Item, if a Play Item changes, a reproducible 
elementary stream also changes. The status transition is aimed to set PSR1 to a most appropriate value for each Play 
Item by executing the "Procedure when playback condition is changed" each time a Play Item starts to be reproduced. 
[0134] In the above-described status transition, the "Procedure when playback condition is changed" is executed as 

45 shown in Fig. 29B. Fig. 29B is a flowchart of the "Procedure when playback condition is changed". The procedure sets 
a value in PSR1 by performing two judgment steps, S1 and S2. 

In step S1, it is judged whether the number of entries in STN_table is "0". If it is judged that the number of entries in 
STN_table is "0", the value of PSR1 is maintained (step S3). 

[0135] In step S2, which is executed if it is judged in step S1 that the number of entries in STN_table is not "0", it is 
50 judged whether the number of entries in STN_table is equal to or larger than PSR1 , and condition (A) is true. Here, the 
condition (A) is that the reproduction apparatus has capability to reproduce a Primary audio stream identified by PSR1 . 
If the judgment result of step S2 is positive, the value of PSR1 is maintained (step S4). If either the value of PSR1 is 
larger than the number of entries in STN_table or the condition (A) is not satisfied, a process is performed to set PSR1 
(step S5). 

55 [0136] Fig. 30 is a flowchart showing the detailed procedure of step S5. 

In this flowchart, steps S6 and S7 constitute a loop in which step S8 is repeatedly performed for all Primary audio 
Streams. In this loop, the process-target Primary audio stream is referred to as Primary audio stream i. In step S8, it is 
checked whether Primary audio stream i satisfies conditions (a), (b), and (c). 
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[0137] The condition (a) is that the reproduction apparatus has capability to reproduce Primary audio stream i, and 
whether the condition is satisfied is judged by comparing PSR15 with stream_coding_type of Primary audio stream i. 
The condition (b) is that the language attribute of Primary audio stream i is the same as the language setting in the 
reproduction apparatus, and whether the condition is satisfied is judged by checking whether audio_language_code of 
Primary audio stream i written in STN_table is equal to the value set in PSR. 

[0138] The condition (c) is that the channel attribute of Primary audio stream i is surround, and the reproduction 
apparatus has capability to reproduce it, and whether the condition is satisfied is judged by comparing PSR15 with 
audio_presentation_type, stream_coding_type of Primary audio stream i. 

Based on the pattern of (i) which conditions, among the plurality of conditions, are satisfied by Primary audio stream i 
and (ii) how many conditions are satisfied, this flowchartassigns priorities to the Primary audio streams. 
After the process is repeatedly performed for all Primary audio streams, steps S9 to S13 are performed. In step S9, it 
is judged whether it is true that no Primary audio stream satisfies condition (a). If it is judged that it is true, an unspecified 
value (OxFF) is set to PSR1 (step S14). 

[0139] In step S1 0, it is judged whether there are Primary audio streams that satisfy all of conditions (a), (b), and (c) . 
If it is judged positively, the stream numbers of such Primary audio streams that satisfy all of conditions (a), (b), and (c) 
are set in PSR1. 

Here, if there are a plurality of Primary audio streams that satisfy all of conditions (a), (b), and (c), it is necessary to 
determine one among them that is set in PSR1 . In step S15, it is determined based on the order of entry in the STN_ 
table. That is to say, in step S1 5, if there are a plurality of Primary audio streams that are the same in the codec, language 
attribute, and channel attribute, the STN_table is referred to and a Primary audio stream that is the first in the order of 
entry in the STN_table is selected. 

[0140] It is therefore possible for the person in charge of authoring to control the priority levels of streams to be 
reproduced, by changing the order of entry in the STN_table. 

If there is no Primary audio stream that satisfies all of conditions (a), (b), and (c), the control goes to step S1 1 in which 
it is judged whether there are Primary audio streams that satisfy conditions (a) and (b). If it is judged that there are 
Primary audio streams that satisfy conditions (a) and (b), a Primary audio stream among these that is the first in the 
order of entry in the STN_table is set in PSR1 (step S16). 

[0141] If there is no Primary audio stream that satisfies all of conditions (a), (b), and (c) and there is no Primary audio 
stream that satisfies conditions (a) and (b), the control goes to step S1 2 in which it is judged whether there are Primary 
audio streams that satisfy conditions (a) and (c) . If it is judged that there are Primary audio streams that satisfy conditions 
(a) and (c), a Primary audio stream among these that is the first in the order of entry in the STN_table is set in PSR1 
(step S17). 

[0142] If there is no Primary audio stream that satisfies all of conditions (a), (b), and (c), there is no Primary audio 
stream that satisfies conditions (a) and (b), and there is no Primary audio stream that satisfies conditions (a) and (c), 
the control goes to step S13 in which it is judged whether there are Primary audio streams that satisfy condition (a). If 
it is judged that there are Primary audio streams that satisfy condition (a), a Primary audio stream among these that is 
the first in the order of entry in the STN_table is set in PSR1 (step S18). 

[0143] Up to now, the "Procedure when playback condition is changed" has been explained. From now on, the "Pro- 
cedure when Stream change is requested" will be explained. Fig. 31 is a flowchart showing the procedure for setting 
PSR1 when stream change is requested. The flowchart of Fig. 31 differs from the flowchart of Fig. 29B in that in some 
places X is used in place of PSR1 . The "X" is a value determined based on the User Operation information output from 
the operation receiving unit 26 or a button command output from the PG decoder 13. 

[0144] In step S19 of the flowchart, it is judged whether the numberof entries in STN_table is equal to or larger than 
X, and condition (A) is true. Here, the condition (A) is that the reproduction apparatus has capability to reproduce a 
Primary audio stream identified by PSR1. It is judged by comparing PSR15 with stream_coding_type of the Primary 
audio stream. If the judgment result of step S1 9 is positive, X is set to PSR1 (step S21 ). 

[0145] If either X is larger than the number of entries in STN_table or the condition (A) is not satisfied, it is judged 
whether X is OxFF. Here, if X is not OxFF, it is considered that the number of the Primary audio stream that the user 
intends to select is invalid, and the value X, which is determined based on the user operation, is disregarded and the 
value set in PSR1 is maintained (step S23). 

If X is OxFF, PSR1 is set (step S24). The processing procedure of step S24 is the same as the procedure shown in Fig. 
30 (Among the steps shown in Fig. 30, the judgment in step S9 is not required in the "Procedure when Stream change 
is requested". This is because in the "Procedure when Stream change is requested", if there is no Primary audio stream 
that satisfies conditions (a), (b), and (c), X, which is set by the user, is not set to PSR1, and the value set in PSR1 is 
maintained) 

[0146] <Functional Construction Detail 3: Procedure Executing Unit 43> 

The Procedure executing unit 43 when necessity for changing a piece of Playltem information to another arises, or when 
the user instructed, through an operation, to change the stream number, executes a predetermined stream selection 
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procedure, and writes a new stream number into PSR14. The reproduction apparatus reproduces the Primary audio 
stream according to the stream number written in PSR14. The setting of PSR14 thus enables a Primary audio stream 
to be selected. 

[0147] The Procedure executing unit 43 brings PSR14 to change in status as shown in Fig. 32A. 
5 Fig. 32A shows the status transition of the value set in PSR1 4. It should be noted here that in Fig. 32A, the term "valid" 
means that the value of PSR14 is equal to or smaller than the number of entries written in the STN_table of the Play 
Item, and is decodable. 

[0148] The term "invalid" means that (a) the value of PSR14 is "0", (b) the value of PSR14 is larger than the number 
of entries written in the STN_table of the Play Item, or (c) decoding is not available even if the number of entries written 
10 in the STN table of the Play Item is in the range from "1" to "32". 

The frames drawn with dotted lines in Fig. 32A indicate a procedure for determining the value of PSR when the status 
changes. The procedure for setting a value to PSR is classified into "Procedure when playback condition is changed" 
and "Procedure when Stream change is requested". 

[0149] The "Procedure when playback condition is changed" is a procedure to be executed when the reproduction 
*5 apparatus has changed in the status due to some event that occurred to the apparatus. 

The "Procedure when Stream change is requested" is a procedure to be executed when the user requests some changing 
(in Fig. 32A, "Change a Stream"). 

The "Procedure when playback condition is changed" and "Procedure when Stream change is requested" are stream 
selection procedures and will be described in detail with reference to the flowcharts. 

20 [0150] The arrows in Fig. 32A symbolically indicate the status transition of PSR. 

The notes attached to the arrows indicate events that trigger each status transition. That is to say, Fig. 32A indicates 
that a status transition of PSR14 occurs if an event such as "Load Disc", "Change a Stream", "Start PlayList playback", 
"Cross a Playltem boundary", or "Change Primary Audio Stream" occurs. It will be under stood by referring to Fig. 32A 
that the above-mentioned procedures are not performed when a status transition of "Invalid -^Invalid" or "Valid^lnvalid" 

25 occurs. On the other hand, status transitions of "Invalid^Valid" and "Valid^Valid" go through the dotted-line frames. 
That is to say, the above-mentioned "Procedure when playback condition is changed" or "Procedure when Stream 
change is requested" is performed when PSR14 is set to be "Valid". 
[0151] The following describes the events that trigger the status transition. 

The event "Load Disc" indicates that the BD-ROM has been loaded in the reproduction apparatus. In such loading, 

30 PSR14 is once set to an unspecified value (OxFF). 

The event "Start PlayList playback" indicates that a reproduction process based on PlayList has been started. When 
such an event occurs, the "Procedure when playback condition is changed" is executed, and PSR1 4 is set to be "Valid". 
[0152] The event "Terminate PlayList playback" indicates that a reproduction process based on PlayList has been 
terminated. It is understood that when such an event occurs, the "Procedure when playback condition is changed" is 

35 not executed, and PSR14 is set to be "Invalid". 

The event "ChangeXXX" indicates that the user requested to change XXX (in Fig. 32A, Stream). If this event occurs 
when PSR1 is "Invalid" (indicated by "cj1" in Fig. 32A), PSR14 is set to a value as requested. Even if the value that is 
set in this way indicates a valid stream number, the value set in PSR14 is treated as an "Invalid" value. That is to say, 
in the status transition triggered by the event "ChangeXXX", PSR never changes from Invalid to Valid. 

40 [0153] On the other hand, if event "Change a Stream" occurs when PSR14 is "Valid" (indicated by "cj2" in Fig. 32A), 
the "Procedure when Stream change is requested" is executed, and PSR14 is set to a new value. Here, the value set 
when the "Procedure when Stream change is requested" is executed may not be a value desired by the user. This is 
because the "Procedure when Stream change is requested" has a function to exclude an invalid value. If the event 
"Change stream" occurs when PSR14 is "Valid", PSR14 never changes from Valid to Invalid. This is because the 

45 "Procedure when Stream change is requested" assures that PSR14 does not become "Invalid". 

[0154] The event "Cross a Playltem boundary" indicates that a Play Item boundary has been passed. Here, the Play 
Item boundary is a boundary between two successive Play Items, namely a position between the end of a preceding 
Play Item and the start of a succeeding Play Item, among the two successive Play Items. If the event "Cross a Playltem 
boundary" occurs when PSR14 is "Valid", the "Procedure when playback condition is changed" is executed. After the 

50 execution of the "Procedure when playback condition is changed", the status of PSR1 4 either returns to "Valid", or if the 
event "CrossaPlayltemboundary" or "ChangePrimaryAudioStream" occurs, transits to "Invalid". The status transition is 
aimed to set PSR14 to a most appropriate value for each Play Item by executing the "Procedure when playback condition 
is changed" each time a Play Item starts to be reproduced or each time a Primary audio stream is changed. 
[0155] In the above-described status transition, the "Procedure when playback condition is changed" is executed as 

55 shown in Fig. 32B. The procedure sets a value in PSR14 by performing two judgment steps, S31 and S32. 

In step S31 , it is judged whether the number of entries in STN_table is "0". If it is judged that the number of entries in 
STN_table is "0", the value of PSR14 is maintained (step S33). 

[0156] In step S32, which is executed if it is judged in step S31 that the number of entries in STN_table is not "0", it 
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is judged whether the number of entries in STN_table is equal to or larger than PSR14, and conditions (A) and (B) are 
true. Here, the condition (A) is that the reproduction apparatus has capability to reproduce a Primary audio stream 
identified by PSR14, and the condition (B) is that the combination of Primary_Audio_Stream_Number and Secondary_ 
Audio_Stream_Number is permitted in the STN_table. If the judgment result of step S32 is negative (NO in step S32), 
the value set in PSR14 is maintained (step S34). If the judgment result of step S32 is positive (YES in step S32), a 
process is performed to set PSR14 (step S35). 

[0157] Fig. 33 is a flowchart showing the detailed procedure of step S35. 

In this flowchart, steps S36 and S37 constitute a loop in which step S38 is repeatedly performed for all Secondary audio 
streams. In this loop, the process-target Secondary audio stream is referred to as Secondary audio stream i. In step 
S38, it is checked whether Secondary audio stream i satisfies conditions (a), (b), and (c). 

[0158] The condition (a) is that the reproduction apparatus has capability to reproduce Secondary audio stream i, and 
whether the condition is satisfied is judged by comparing the register indicating the audio stream reproduction capability 
(PSR15) with stream_coding_type of Secondary audio stream i. 

The condition (b) is that the Primary audio stream can be mixed with the Secondary audio stream, and whether the 
condition is satisfied is judged by checking whether the stream number set in PSR14 is written in Comb_info_Secondary_ 
audio_Primary_audio. 

[01 59] The condition (c) is that the language attribute of Secondary audio stream i is the same as the language setting 
in the reproduction apparatus, and whether the condition is satisfied is judged by checking whether audio_language_ 
code of Secondary audio stream i written in STN_table is equal to the value set in PSR. 

Based on the pattern of (i) which conditions, among the plurality of conditions, are satisfied by Secondary audio stream 
i and (ii) how many conditions are satisfied, this flowchart assigns priorities to the Secondary audio streams. 
After the process is repeatedly performed for all Secondary audio streams, steps S39 to S41 and steps S44 to S46 are 
performed. In step S39, it is judged whether it is true that no Secondary audio stream satisfies conditions (a) and (b). If 
it is judged that it is true, an unspecified value (OxFF) is set to PSR14 (Step S44). 

[0160] In step S40, it is judged whether there are Secondary audio streams that satisfy all of conditions (a), (b), and 
(c). If it is judged positively, the stream numbers of such Secondary audio streams that satisfy all of conditions (a), (b), 
and (c) are set in PSR14. 

Here, if there are a plurality of Secondary audio streams that satisfy all of conditions (a), (b), and (c), it is necessary to 
determine one among them that is set in PSR14. In step S45, it is determined based on the order of entry in the STN 
table. That is to say, in step S45, if there are a plurality of Secondary audio streams that are the same in the codec, 
language attribute, and channel attribute, the STN_table is referred to and a Secondary audio stream that is the first in 
the order of entry in the STN_table is selected. 

[0161] It is therefore possible for the person in charge of authoring to control the priority levels of streams to be 
reproduced, by changing the order of entry in the STN_table. 

If there is no Secondary audio stream that satisfies all of conditions (a), (b), and (c), the control goes to step S41 in 
which it is judged whether there are Secondary audio streams that satisfy conditions (a) and (b). If it is judged that there 
are Secondary audio streams that satisfy conditions (a) and (b), a Secondary audio stream among these that is the first 
in the order of entry in the STISMable is set in PSR14 (step S46). 

[0162] Up to now, the "Procedure when playback condition is changed" has been explained. From now on, the "Pro- 
cedure when Stream change is requested" will be explained. Fig. 34 is a flowchart showing the procedure for setting 
PSR14 when stream change is requested. The flowchart of Fig. 34 differs from the flowchart of Fig. 32B in that in some 
places X is used in place of PSR14. The "X" is a value determined based on the User Operation information output from 
the operation receiving unit 26 or a button command output from the PG decoder 13. 

[0163] In step S49 of the flowchart, it is judged whether the number of entries in STN_table is equal to or larger than 
X, and conditions (A) and (B) are true. If the judgment result of step S49 is positive, X is set to PSR14 (step S51). 
If either X is larger than the number of entries in STN_table or the conditions (A) and (B) are not satisfied, it is judged 
whether X is OxFF (step S52). Here, if X is not OxFF, it is considered that the number of the Secondary audio stream 
that the user intends to select is invalid, and the value X, which is determined based on the user operation, is disregarded 
and the value set in PSR14 is maintained (step S53). 

[0164] If X is OxFF, PSR14 is set (step S54). The processing procedure of step S54 is the same as the procedure 
shown in Fig. 33 (Among the steps shown in Fig. 30, the judgment in step S39 is not required in the "Procedure when 
Stream change is requested" . This is because in the "Procedure when Stream change is requested", if there is no 
Primary audio stream that satisfies conditions (a), (b), and (c), X, which is set by the user, is not set to PSR14, and the 
value set in PSR14 is maintained) 

[0165] This completes the explanation of the Procedure executing unit 43. 
<Functional Construction Detail 4: Mixing Control Unit 44> 

The mixing control unit 44, if the Player Profile information is 0001 h or 001 1 h, controls the mixers 9a and 9b to mix the 
reproduction output of the Primary audio stream with the reproduction output of the Secondary audio stream or the 
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reproduction output of the sound data. 

[0166] When the current reproduction point in the Playltem time axis is in a period from In time to Out time of the 
SubPlayltem information, and when the Secondary audio stream is valid in the STN_Table of the current Playltem 
information, the mixing control unit 44 controls the mixer 9a to mix the reproduction output of the audio decoder 7a with 
5 the reproduction output of the audio decoder 7b because the audio decoder 7b is decoding the Secondary audio stream 
having the stream number stored in PSR14. 

[0167] If the attribute of the Primary audio stream is "surround", the reproduction output of the Secondary audio stream 
can be mixed after a downmixing is performed such that only desired components are held among components such 
as L, R, C, LS, RS, LR, RR, and LFE. Here, when the Secondary audio stream is, for example, a commentary of the 
to movie director, by changing the channel of the Primary audio stream, with which the Secondary audio stream is to be 
mixed, from L to C and to R, it is possible to give the user a sensation as if the director is waking around the user. Such 
mixing is called panning. In panning, the sound/voice of a Secondary audio stream (for example, monaural) with a less 
number of channels than a Primary audio stream is used. 

[01 68] When a confirmation operation is performed on a button drawn by a Java ™ application, or when a confirmation 
*5 operation is performed on a button drawn by an IG stream, the mixing control unit 44 controls the mixer 9b to mix the 
sound data with the reproduction output of the Primary audio stream or with the mixing result of the reproduction output 
of the Primary audio stream and the reproduction output of the Secondary audio stream. 

[0169] As described above, according to the present embodiment, the demultiplexer 3b can obtain a TS packet that 
is transferred on the bus 1b, refer to the upper byte of the PID, and enter the TS packet into a decoder dedicated to 
20 Secondary audio streams. With this construction, the demultiplexer 3b can demultiplex only the PES packets that con- 
stitute the Secondary audio stream, and enter the PES packets into a decoder that is different from a decoder dedicated 
to Primary audio streams. 

[0170] Since it is possible to enter packets that constitute a Primary audio stream and packets that constitute a 
Secondary audio stream respectively into different decoders, it is possible, for example, to downconvert, downmix, or 
25 reduce the gain of only the packets that constitute a Primary audio stream. This makes it possible to subject only the 
packets that constitute a Primary audio stream to special processes in real time even when reading out a Secondary 
audio stream from the local storage 200 while reading out the Primary audio stream from the BD-ROM 1 00 and supplying 
the two streams to the two respective decoders. 
Second Embodiment 

30 The second embodiment of the present invention relates to the selection of an apparatus to decode a Primary audio 
stream among the reproduction apparatus 300 and the AV amplifier 500 in a home theater system. 
[0171] Fig. 35A shows the connection form of the reproduction apparatus 300, the AV amplifier 500, and the speakers 
600. The connection form in the home theater system is classified into a form in which the reproduction apparatus 300 
is directly connected to the speakers 600 without connection to the AV amplifier 500 in between (connection form (1)), 

35 and a form in which the reproduction apparatus 300 is connected to the speakers 600 via the AV amplifier 500 (connection 
form (2)). 

In the former connection form (1), the LPCM obtained as a result of decoding by the reproduction apparatus 300 is 
supplied to the speakers 600. On the other hand, in the latter connection form (2), the reproduction apparatus 300 
performs merely a pass-through output, and the LPCM obtained as a result of decoding by the AV amplifier 500 is 
40 supplied to the speakers 600. 

[01 72] The number of channels changes in each apparatus in the home theater system when two or more connection 
forms are available in the home theater system. 

Fig. 35B shows how the number of channels changes in each of the reproduction apparatus 300, the AV amplifier 500, 
and the speakers 600. The 1 st row of Fig. 35B shows each of the BD-ROM 100, the reproduction apparatus 300, the 
45 AV amplifier 500, and the speakers 600. In regards with the decoding capability of these apparatuses, the reproduction 
apparatus 300 is capable of decoding up to 5.1 channels, the AV amplifier 500 is capable up to 7.1 channels, and the 
speakers 600 is capable up to 7.1 channels . 

[0173] The 2 nd to 4 th rows show by which apparatus and how the downmixing is performed when the BD-ROM 100 
has Primary audio streams of 7.1 channels and 5.1 channels. The 2 nd row shows the case where the BD-ROM 1 00 has 

50 a Primary audio stream of 7. 1 channels and it is decoded by the reproduction apparatus 300. In this case, the reproduction 
apparatus 300 reads out the Primary audio stream of 7.1 channels from the BD-ROM 100, decodes and downmixes it 
to 5 .1 channels, and outputs an LPCM of the downmixed 5.1 channels . The AV amplifier 500 converts the LPCM of 
the downmixed 5.1 channels to 7.1 channels, and outputs the converted LPCM of 7.1 channels to the speakers 600. 
[0174] The 3 rd row shows the case where the BD-ROM 1 00 has a Primary audio stream of 5.1 channels. In this case, 

55 the reproduction apparatus 300 decodes the Primary audio stream of 5.1 channels, and outputs an LPCM as the decoding 
result to the AV amplifier 500 without downmixing. The AV amplifier 500 converts the LPCM of 5.1 channels to 7.1 
channels, and outputs the converted LPCM of 7.1 channels to the speakers 600. 

The 4 th row shows the case where the BD-ROM 1 00 has a Primary audio stream of 7.1 channels and it is output by the 
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reproduction apparatus 300 by the pass-through output. In this case, the decoding is performed by the AV amplifier 500, 
and the sound/voice of 7.1 channels, which is the decoding result of the AV amplifier 500, is supplied to the speakers 
600 and is output as the sound/voice. 

[0175] These outputs are compared as follows. In the case of the 2 nd row, the reproduction apparatus 300 converts 
5 the sound/voice from 7.1 channels to 5.1 channels, then the AV amplifier 500 converts it from 5.1 channels to 7.1 
channels, and outputs the sound/voice of 7.1 channels to the speakers 600. With this construction, although the audio 
stream recorded on the BD-ROM 100 is 7.1 channels, the audio quality of the sound/voice finally output in the case of 
the 2 nd row hardly differs from the audio quality of the sound/voice finally output in the case of the 3 rd row. 
[0176] On the other hand, in the case of the 4 th row, since the AV amplifier 500, which has the decoding capability of 
to 7.1 channels, is caused to decode the audio stream of 7.1 channels, a high-quality output is expected. 

In the present embodiment, in such an output form, when a device to which the reproduction apparatus is connected to 
has the reproduction capability, the reproduction apparatus performs the pass-through output so that the device performs 
the decoding. 

The DIB (Decoder Information Block) of the HDMI is used to determine how high the reproduction capability or the 
15 decoding capability of the connection partner device is, or to determine whether to perform a stream output. Fig. 36A 
shows the DIB. 

[0177] As shown in Fig. 36A, the DIB includes "Coding Type" that shows the encoding method by which an output to 
the AV amplifier 500 is possible, "Channel Count" that shows the number of channels with which the connection partner 
device can decode, "Channel/Speaker Allocation" that shows the arrangement of speakers, and "Sample Frequency" 
20 that shows a frequency at which the connection partner device can reproduce. 
Fig. 36B shows the values that can be set in each field of the DIB. 

[0178] In the example of Fig. 36B, the row of the "Coding Type" shows encoding methods DTS-HD, MLP, and DD+by 
which the connection partner device can decode. In the actual writing of the "Coding Type", four parameters "CT0", 
"CT1", "CT2", and "CT3" are used. Also, the row of the "Channel Count" shows 5.1 ch, 7.1ch, and 2ch which are the 

25 numbers of channels with which the connection partner device can decode. 

Also, the row of the "Channel/Speaker Allocation" shows "L/R/C/LS/RS/LFE", "L/R/C/LS/RS/LR/RR/LFE", and "L/R" 

which indicate physical arrangements of speakers for 5.1 ch, 7.1ch, and 2ch, respectively. 

[0179] The row of the "Sample Frequency" shows sampling frequencies "48KHz", "192KHz", and "96KHz". 

Fig. 36C shows a communication sequence using the DIB between the reproduction apparatus 300 and the AV amplifier 

30 500. 

When the reproduction apparatus 300 is connected with the AV amplifier 500, the HDMI transmission/reception unit 27 
of the reproduction apparatus 300 performs a mutual authentication with the AV amplifier 500 as indicated by "®1 ", then, 
as indicated by "®2", receives the DIB from the AV amplifier 500, which is to be a receiver. If, upon referring to the DIB, 
it is confirmed that the AV amplifier 500 has the capability to decode the Primary audio stream, the reproduction apparatus 
35 300 can output the Primary audio stream to the AV amplifier 500 by the pass-through output, as indicated by "©3". If 
the AV amplifier 500 does not have the capability to decode the Primary audio stream, either the LPCM decoded by the 
reproduction apparatus is transferred, or the stream is not be reproduced. 

[0180] The following describes an improvement to the reproduction apparatus in the second embodiment. 
Fig. 37 shows an internal structure of PSR15 in the PSR set 23. 

40 PSR15 is 32-bit long as is the case with PSR1, PSR14, and PSR31 explained in the first embodiment. 

The bits bO to b3 show whether the reproduction apparatus has a capability to decode and reproduce LPCM audio 
streams. When the four bits are "0001 "b, it indicates that the reproduction apparatus has a capability to reproduce LPCM 
audio streams of 48/96Hz having the stereo attribute. When the four bits are "001 0"b, it indicates that the reproduction 
apparatus has a capability to reproduce LPCM audio streams of 48/96Hz having the surround attribute. When the four 

^5 bits are "01 01 "b, it indicates that the reproduction apparatus has a capability to reproduce LPCM audio streams of any 
frequency having the stereo attribute. When the four bits are "01 10"b, it indicates that the reproduction apparatus has 
a capability to reproduce LPCM audio streams of any frequency having the surround attribute. 

[0181] The bits b4 to b7 of PSR 15 show whether the reproduction apparatus has a capability to decode and reproduce 
DD/DD+ audio streams. When the lower two bits of the four bits are "01 "b, it indicates that the reproduction apparatus 
50 has a capability to reproduce up to two channels (2ch) of basic data (independent substreams) of DD/DD+ audio streams. 
When the lower two bits of the four bits are "1 0"b, it indicates that the reproduction apparatus has a capability to reproduce 
multi channels (channels that are larger in number than 2ch) of basic data (independent substreams) of DD/DD+ audio 
streams. 

[0182] When the upper two bits of the four bits are "01 "b, it indicates that the reproduction apparatus has a capability 
55 to reproduce up to two channels (2ch) of extension data (dependent substreams) of DD/DD+ audio streams. When the 
upper two bits of the four bits are "1 0"b, it indicates that the reproduction apparatus has a capability to reproduce multi 
channels of extension data (dependent substreams) of DD/DD+ audio streams. 

[0183] When the upper two bits of the four bits are "00"b, it indicates that the reproduction apparatus does not have 
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a capability to reproduce extension data (dependent substreams) of DD/DD+ audio streams. 

The bits b8 to b1 1 of PSR15 show whether the reproduction apparatus has a capability to decode and reproduce DTS- 
HD audio streams. When the lower two bits of the four bits are "01 "b, it indicates that the reproduction apparatus has a 
capability to reproduce up to two channels (2ch) of basic data (Core substreams) of DTS-HD audio streams. When the 
lower two bits of the four bits are "10"b, it indicates that the reproduction apparatus has a capability to reproduce multi 
channels of basic data (Core substreams) of DTS-HD audio streams. 

[01 84] When the upper two bits of the four bits are "01 "b, it indicates that the reproduction apparatus has a capability 
to reproduce up to two channels (2ch) of extension data (Extension substreams) of DTS-HD audio streams. When the 
upper two bits of the four bits are "1 0"b, it indicates that the reproduction apparatus has a capability to reproduce multi 
channels of extension data (Extension substreams) of DTS-HD audio streams. 

When the upper two bits of the four bits are "00"b, it indicates that the reproduction apparatus does not have a capability 
to reproduce extension data (Extension substreams) of DTS-HD audio streams. 

[01 85] The bits b1 2 to b1 5 of PSR1 5 show whether the reproduction apparatus has a capability to decode and reproduce 
DD/MLP audio streams. When the lower two bits of the four bits are "01 "b, it indicates that the reproduction apparatus 
has a capability to reproduce up to two channels (2ch) of DD audio streams. When the lower two bits of the four bits are 
"10"b, it indicates that the reproduction apparatus has a capability to reproduce multi channels of DD audio streams. 
[01 86] When the upper two bits of the four bits are "01 "b, it indicates that the reproduction apparatus has a capability 
to reproduce up to two channels (2ch) of MLP audio streams. When the upper two bits of the four bits are "10"b, it 
indicates that the reproduction apparatus has a capability to reproduce multi channels of MLP audio streams. 
When the upper two bits of the four bits are "00"b, it indicates that the reproduction apparatus does not have a capability 
to reproduce MLP audio streams. 

[01 87] As described above, PSR1 5 can define, for each encoding method, whether or not the reproduction apparatus 
has a capability to process the basic data and the extension data. Here, the "2ch" used in the explanation of PSR15 
means two channels for L and R, respectively. 

With such definition of PSR15, the controller 22 in the second embodiment performs the reproduction control of the 
Primary audio stream, in accordance with the flowchart shown in Fig. 38. Fig. 38 is a flowchart showing the processing 
procedure of the controller 22 in the second embodiment. 

[0188] In step S61 , it is judged whether the apparatus has been activated. If it is judged that the apparatus has been 
activated, the partner apparatus connected via HDMI is authenticated (step S62). When the connected apparatuses are 
successfully authenticated mutually, the control moves to step S63. The step S63 is a loop in which it is repeatedly 
judged whether or not the DIB has been received. If it is judged that the DIB has been received (YES in step S63), the 
controller 22 recognizes, by referring to the DIB received from the connection partner, the reproduction capability and 
the number of speakers (arrangement) of the connection partner (step S64). Then in step S65, it is judged whether or 
not the connection partner has the reproduction capability. If it is judged that the connection partner has the reproduction 
capability, the controller 22 displays a menu for receiving a request for the pass-through output (step S67), and judges 
whether the user requested the pass-through output (step S68). 

It should be noted here that if it is recognized, by referring to the DIB, that a device (in many cases, an AV amplifier) 
connected to a speaker has the decoding capability, the pass-through output may be set automatically (YES in step 
S68) so that the connection partner device performs the decoding, without allowing the user to set. 
[0189] Further, it may be preliminarily set that if it is recognized, by referring to the apparatus settings menu (initial 
settings menu) of the reproduction apparatus 300, that the connection partner device has the decoding capability, the 
pass-through output is set automatically (YES in step S68). 

Figs. 39 and 40 shows menus for receiving a determination on whether or not to perform the pass-through output. The 
menus shown in Figs. 39 and 40 are each composed of a message that is supposed to attract the user' s attention: "In 
your theater system, audio decoding is available at a connection-destination device"; or "You can enjoy high-quality 
audio when audio decoding is done at a connection-destination device XXX", and buttons "YES" and "NO" for receiving 
determination on whether or not to allow the connection-destination device to perform the decoding. 
[0190] In the message shown in the menu of Fig. 40, "XXX" may be filled with the product name or the model number 
of the AV amplifier 500. Alternatively, "XXX" may be filled with the product name or the model number of the television 
400 as well as those of the AV amplifier 500, allowing the user to select one among them. This enables the user to select 
a device that is to perform the decoding. Further, if the user has two or more AV amplifiers, it is preferable that the 
product names and/or the model numbers of the AV amplifiers are displayed such that the user can select one among them. 
[01 91 ] When the user makes a confirmation on the "YES" button on the menu to instruct the pass-through output, the 
Player capability for Audio is set in PSR15 according to the DIB (step S71), and the Procedure executing unit 42 is 
caused to execute the procedure of selecting a Primary audio stream according to the decoding capability of the AV 
amplifier 500 (step S72). After this, the Primary audio stream of the AVClip read out from the BD-ROM is output by the 
pass-through output (step S73). 

[0192] If the user does not operate to cause the AV amplifier 500 to perform the decoding, the controller 22 sets PSR1 5 
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in accordance with the decoding capability of the reproduction apparatus itself, causes the Procedure executing unit 42 
to execute the procedure of selecting a Primary audio stream (step S69), and causes the sound/voice (LPCM) decoded 
by the reproduction apparatus to be output (step S70). 

Figs. 41 and 42 are flowcharts showing the procedures for setting Player capability for Audio in PSR1 5 according to the 
DIB. 

[0193] In the flowchart of Fig. 41 , in step S200, it is judged whether or not DIB.CODING_TYPE indicates DD+. 

If it is judged that DIB.CODING_TYPE indicates DD+, the control moves to step S201 in which it is judged whether or 

not the Channel count of the DIB has a value larger than 2. If the judgment result of step S201 is positive, the control 

moves to step S203 in which the "capability" of Dependent SubStream is set to 10b:Surround Capable. 

[0194] If it is judged as "NO" in step S201, the control moves to step S205 in which the "capability" of Dependent 

SubStream is set to 01b:Stereo Capable. 

In the flowchart of Fig. 42, in step S82, it is judged whether or not DIB.CODING_TYPE indicates MLP. If it is judged that 
DIB.CODING_TYPE indicates MLP (YES in step S82), the control moves to step S91 in which it is judged whether or 
not the Channel count of the DIB has a value larger than 2. 

[0195] If the judgment result of step S91 is positive (YES in step S91), the control moves to step S93 in which the 
"capability" of MLP is set to 1 0b:Surround Capable. If it is judged as "NO" in step S91 , the control moves to step S94 in 
which the "capability" of MLP is set to 01b:Stereo Capable. 

It should be noted here that although Channel Count of the DIB is used in the judgments in steps S201 and S91 of Figs. 
41 and 42, Channel/Speaker Allocation may be used instead. 

[0196] As described above, according to the present embodiment, information of the capability of the connection 
partner device is obtained via a digital interface such as HDMI, and PSR15 is set according to the obtained capability 
information. Further, a Primary audio stream is selected from the BD-ROM disc/local storage, and the pass-through 
output of the selected Primary audio stream is performed. This makes it possible to effectively use the decoder resource 
of the connection partner device to achieve a high-quality audio reproduction. Also, since an AV amplifier and speakers 
are often developed/sold as a set. In such a case, the AV amplifier provides adjustment that matches the characteristics 
of the speakers. Accordingly, even if the reproduction apparatus can decode using the internal decoder, it is possible 
to cause the AV amplifier connected to the reproduction apparatus to decode to obtain a reproduction result of an 
acoustically higher quality. 
[0197] Third Embodiment 

The local storage 200 in the first embodiment is a hard disk embedded in the reproduction apparatus, while the local 
storage 200 in the third embodiment is an SD memory card which is a portable recording medium. The present embod- 
iment relates to an improvement in the case where a dynamic scenario is recorded in the portable recording medium 
for use in the reproduction apparatus. 

[0198] The SD memory card is a portable recording medium that is provided with a secure area and a non-secure 
area. Files on a local storage such as those shown in the first embodiment are recorded in the non-secure area after 
they are encoded, and the encryption key is recorded in the secure area. The encryption key cannot be read out from 
the secure area unless the authenticity of the reader is confirmed through the authentication performed by the SD memory 
card. This construction prevents an unauthorized copy of the files from the non-secure area, and enables the files to be 
read out only when the BD-ROM is accessible as well. 

[0199] The dynamic scenario is a program such as a Java ™ application or a Movie Object described in the first 
embodiment that causes the reproduction apparatus to perform a reproduction via the PlayList information. The dynamic 
scenario may be recorded on the BD-ROM 1 00 or on the local storage 200. When the dynamic scenario is recorded on 
a portable recording medium, it means that the dynamic scenario is accessible by any of reproduction apparatuses 
disposed at various locations. Accordingly, the dynamic scenario may be accessed by the reproduction apparatus 300 
that is mounted in a car. 

[0200] Fig. 43A shows the reproduction apparatus 300 mounted in a car. The reproduction apparatus 300 and the 
television 400 constitute acar audio system. The portable local storage 200 may be installed in the reproduction apparatus 
300, together with the BD-ROM 100. 

When the reproduction apparatus 300 is mounted in a car, it is preferable to clearly indicate that the reproduction 
apparatus 300 is operated like a CD reproduction apparatus. To clearly indicate that the reproduction apparatus 300 is 
operated like a CD reproduction apparatus, PSR31 in the present embodiment is constructed as shown in Fig. 43B. 
[0201] Fig. 43B shows the internal structure of PSR31 in the third embodiment. When the reproduction apparatus is 
mounted in a car, the Player Profile information of PSR31 in the third embodiment is set to "1000"b by an engineer of 
the maker or by an end user. When Profile information of PSR31 is set to "1 000"b, the Profile information indicates that 
the reproduction apparatus is operated as an Audio Profile Player. Here, the "Audio Profile Player" indicates that the 
reproduction apparatus is not suited for being operated via an interactive screen, but is operated CD-player-like (that is 
to say, the reproduction apparatus is a BD-ROM reproduction apparatus that performs a reproduction control on the 
contents only by linear operations like a CD reproduction apparatus, and is not ready for a reproduction control on the 
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contents via a GUI or the like, or the reproduction apparatus temporarily transfers into such a mode when it detects that 
the car is running). 

[0202] When the reproduction apparatus is CD-player-like, the content is reproduced by operations that are performed 
in units of chapters, such as a chapter skip and a chapter search. The chapter is defined by the PlayList Mark information 
shown in Fig. 17. 

The following describes the PlayList Mark information. 

Fig. 44 shows the internal structure of the PlayList Mark information contained in the PlayList information. As indicated 
by the lead line pmO in the drawing, the PlayList Mark information includes a plurality of pieces of PL Mark information 
(#1-#n). The PL Mark information (PL Mark()) specifies a given position on the PL time axis as a chapter point. As 
indicated by the lead line pm1 in the drawing, the PL Mark information includes: "ref_to_Playltem_ld" that indicates a 
Playltem being a target of the chapter specification; and "Mark_time_stamp" that indicates a chapter position in the 
Playltem by a time notation. 

[0203] Fig. 45 shows a specification of a chapter position by the PL Mark information. The 2 nd to 5 th rows in Fig. 45 
show the EP_map and AVClip shown in Fig. 10. 

The 1 st row shows the PL Mark information and the PL time axis. Two pieces of PL Mark information #1 and #2 are 
shown in the 1 st row. The arrows kt1 and kt2 indicate specifications of Playltems by ref_to_Playltem_ld in the PL Mark 
information. As understood from these arrows, ref_to_Playltem_ld in the PL Mark information specifies Playltems to be 
referred to. Also, the Mark_time_stamp indicates the times of Chapters #1 and #2 on the PL time axis. In this way, the 
PL Mark information defines chapter points on the Playltem time axis. 

[0204] Up to now, the PL Mark information has been described. From now on, the improvement on the dynamic 
scenario will be described. 

In the case of a program that is embedded in the reproduction apparatus, it is known in the manufacturing stage whether 
the program is used in a car or in a home theater system. However, in the case of the dynamic scenario (Movie Object, 
Java ™ application) that is supplied to the reproduction apparatus via the BD-ROM 100 or the portable local storage 
200, it cannot be known preliminarily whether the program is used in a car or in a home theater system. This is because 
the user determines what reproduction apparatus the BD-ROM 100 or the local storage 20 being an SD memory card 
should be used in. 

[0205] For this reason, when the Player Profile information is set to specify Audio Profile Player, the Movie Object or 
the Java ™ application needs to control the reproduction apparatus to be operated CD-player-like. On the other hand, 
when the Player Profile information is set otherwise, it is preferable that the reproduction apparatus is operated via an 
interactive screen. 

Fig. 46 is a flowchart of a processing procedure of a Java ™ application in the present embodiment. 
[0206] In step S149, it is judged whether or not PSR31 is set to specify Audio profile player. If the judgment result in 
step S1 49 is negative, the PlayList starts to be reproduced (step S1 50). The control then moves to the loop in step S1 51 . 
The step S151 constitutes a loop waiting for a Menu call operation. If it is judged that the Menu call operation has been 
performed, the menu is displayed (step S1 52), and steps S1 53 to S1 56 are performed. In step S1 53, it is judged whether 
or not a Focus move operation has been performed. If it is judged that the Focus move operation has been performed, 
the focus is moved to another button (step S155). 

[0207] In step S154, it is judged whether or not an operation has been performed to make confirmation on the button. 
If it is judged that the operation has been performed to make confirmation on the button, a process is executed in 
correspondence with the button (step S156). 

If it is judged in step S149 that PSR31 is set to specify Audio profile player (YES in step S149), a PlayList of an audio 
announcement is reproduced to notify the user that a linear CD-like reproduction is performed. The control then moves 
to a loop composed of steps S158 to S159. 

[0208] In step S158, it is judged whether or not a numeric has been input. In step S159, it is judged whether or not 
the SkipNext key or the SkipBack key has been operated. 

If it is judged in step S158 that a numeric has been input, the control moves to step S160 in which the chapter search 
function is executed. If it is judged in step S1 59 that the SkipNext key or the SkipBack key has been operated, the control 
moves to step S161 in which the chapter skip function is executed. 

[0209] In the above-provided example, a Java ™ application is used. When a Movie Object is used, steps S153 to 
S155 are performed using navigation commands in the IG stream or navigation commands of the Movie Object. The 
operations in this case are similar to those for a Java™ application, and the explanation is omitted. 
The following provides explanation of the procedures of the chapter search and the chapter skip, with reference to the 
flowcharts. Fig. 47 is a flowchart showing the procedure of the chapter search. 

[0210] First, the PL Mark information corresponding to an input value is set to the current PL Mark (step S125). 
Secondly, in step S126, the PI written in ref to Playltemjd of the Current PL Mark is set to Playltem#x (step S126). In 
step S127, the Clip information specified by Clip information file name of Playltem#x is read out. In step S128, mark_ 
time_stamp of the current PL Mark is converted into l-Picture address u using EP_map of current Clip information. 
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[0211] In step S129, Out_time of Playltem#x is converted into 1 -Picture address v using EP_map of the current Clip 
information. In step S130, the decoder is instructed to output the data extending from mark_time_stamp of the current 
PLMark to Out time of Playltem#x. Up to now, the chapter search has been described. From now on, the chapter skip 
will be explained. Fig. 48 is a flowchart showing the procedure of the chapter skip. 

5 [0212] In step S132, it is judged which of the SkipNext key and the SkipBack key was depressed. If it is judged that 
the SkipBack key was depressed, the control moves to step S133 in which the direction flag is set to -1 . If it is judged 
that the SkipNext key was depressed, the control moves to step S134 in which the direction flag is set to +1 . 
In step S135, the current PI number and the current PTM are converted to specify the current PL Mark. In step S136, 
a result of adding the value of the direction flag to the current PL Mark number is set to the current PL Mark number. 

to Here, in the case of the SkipNext key, the direction flag has been set to +1, thus the current PL Mark number is 
incremented; and in the case of the SkipBack key, the direction flag has been set to -1 , thus the current PL Mark number 
is decremented. After the PL Mark information is set in this way, TS packets are read out by executing steps S126 to 
S130, as in Fig. 47. 

[0213] Up to now, the procedure of the reproduction apparatus in a reproduction based on the PL Mark information 
is has been explained. 

As described above, according to the present embodiment, when the PSR31 is set to specify Audio Profile Player, a 
dynamic scenario such as Movie Object or Java ™ application can be used so that the user can reproduce a PlayList 
with operationability equivalent to that of a CD reproduction apparatus. With such a construction, the BD-ROM can be 
operated by the user in the same manner as a music CD. More specifically, the chapter search/skip function corresponds 

20 to the function to search/skip songs in the music CD. Also, when a dynamic scenario runs on a reproduction apparatus 
that is not set to Audio Profile Player, an interactive reproduction is available, with a GUI-based menu being displayed 
on presumption that images are output. In the case where reproduction is performed via an interface such as HDMI that 
enables the state of the connected devices to be monitored, the following are available. That is to say, it is possible to 
monitor the ON/OFF of the power of the monitor display or to monitor the selection of an input source image, and if it is 

25 detected that the monitor display is not displaying reproduced images, PSR31 may be set to specify Audio Profile Player, 
or the reproduction may be paused, or the reproduction apparatus 300 may be powered off in conjunction with the 
monitor display. 
[0214] Fourth Embodiment 

The fourth embodiment relates to how to acquire the PlayList information, Clip information, or AVClip information de- 
30 scribed in the first embodiment. The central body of the acquisition process is a Java ™ application. This is because 

the Java ™ application has ability to form a connection on the network and file a desired file from the server apparatus. 

The connection for acquiring a file from a server can be established using the openConnection method provided in the 

Java ™ software library called Java ™. net. The following is an example of such a connection when the URL of the file 

in the server is http://xxx.eom/vp_data/0.m2ts. 
35 Example: 

new java ™.net.URL(http://xxx.com/vp_data/0.m2ts. 
openConnection() 

40 Fig. 49 is a flowchart of a Java ™ application in the fourth embodiment. In step S181, a PlayList in BD-ROM starts to 
be reproduced. After this, in step S1 82, "Organization" and "Disc-id" are read out from the BD-ROM, where "Organization" 
is information indicating a corporation or an organization that is the distributor of the BD-ROM, and "Disc-id" is an identifier 
of the BD-ROM. 

[0215] In step S183, it is judged whether or not it is true that the local storage 200 has no area corresponding to 
45 Organization or Disc-id read out in step S1 82. 

If it is judged that the local storage 200 does not have such an area, the control moves to step S 184 in which it is judged 
whether or not it is true that Player Profile of PSR31 is not "0000"b. 

[021 6] The current process ends if the judgment result in step S1 83 or step S1 84 is "NO". If the local storage 200 has 
no area corresponding to Organization or Disc-id, and if the audio mixing can be performed, judgment results in both 
50 step S1 83 and step S1 84 are "YES". 

If the judgment results in both step S1 83 and step S1 84 are "YES", a menu having an "update button" is displayed (step 
S185). Then, a confirmation on the "update button" is waited for (step S186). 

[0217] Fig. 50 shows an example of displaying a menu having an "update button". In Fig. 50, the button "Update" is 
the above-indicated "update button". If an operation is performed to make confirmation on the "update button", Disc-id 
55 and PlayList-id are transmitted to the server apparatus in step S1 87. 

The Disc-id is identification information of the BD-ROM and the PlayList-id identification information of the PlayList being 
reproduced. The server apparatus uses these information to select an additional content to be acquired, on the database 
of the contents provider. The selected additional content is sent back to the reproduction apparatus via a network. The 
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additional content includes a set of files (PlayList information, Clip information, and AVClip) to be recorded on the local 
storage 200. 

[0218] In step S188, reception of the additional content is waited for. If the additional content is received, the control 
moves to step S1 89 in which the PlayList information, Clip information, and AVClip that constitute the received additional 

5 content are written into the local storage 200. After this, if downloading from the server apparatus is completed and 
mixing has become available in the reproduction, the user is notified that mixing is available in the reproduction (step 
S190). Then the process enters a loop waiting for the user to request mixing (step S191). If the user requests mixing, 
the control moves to step S1 92 in which the reproduction of the PlayList is stopped, and the virtual package is constructed. 
[0219] Asking the server whether or not an update is available and downloading the content for the update can be 

to executed any time. For example, they can be executed at regular intervals while the disc is loaded or while a content is 
reproduced. Also, the user may be asked whether to reproduce the content immediately after the reproduction is ready 
to start (immediately after the virtual package is ready to be reproduced). 

As described above, according to the present embodiment, when a PlayList is reproduced initially, an AVClip is reproduced 
while a downloading is performed, and the user can watch a reproduced image of the PlayList while waiting for the data 

*5 to be downloaded into the local storage 200. This prevents the user from getting bored of waiting for the data to be 
downloaded into the local storage 200 even if the data has several hundreds mega bytes as a file in the local storage 200. 
[0220] Here, a condition for deleting the additional content from the local storage 200 may be set in the reproduction 
apparatus 300 as an initial setting, and the reproduction apparatus 300 may delete the additional content in accordance 
with the initial setting. For example, it is preferable that the initial setting specifies receiving an input of time, and deleting 

20 the additional content at a predetermined time (for example, automatically deleting the additional content after one month 
has passed). 

Alternatively, when the BD-ROM is ejected, the user may be asked whether to delete the additional content from the 
local storage 200, and the additional content may be deleted if the user answers positively. 

[0221] Further, when the BD-ROM is a partial disc having a read-only area and a writable area, the additional content 
25 in the local storage 200 may be written to the writable area of the partial disc. 
Supplementary Notes> 

Up to now, the best mode for carrying out the invention, as far as known to the applicant at this time of filing the present 
application, has been described. However, further improvements or modifications can be made on the present invention 
in terms of the following technical topics. It should be noted here that whether or not to make such improvements or 
30 modifications is optional, and depends on the implementer of the invention. 
[0222] <Arrangement to Be Made on Additional Contenb* 

It is preferable that the reproduction apparatus is initially set to delete the additional content from the local storage 200 
months or years after the download therein. 
Substitute for PID> 

35 in the first embodiment, the PID is used to discern between the Primary audio stream and the Secondary audio stream. 
In case MPEG2-PS is used instead, it is preferable that PES packet headers of the Primary audio stream and the 
Secondary audio stream have different stream_ids. 

[0223] It is only required that the Primary audio stream and the Secondary audio stream are discernable in the level 
of system streams so that two audio streams are discernable by one demultiplexer. Alternatively, it is only required that 
40 before two streams are combined into one, the PID of one of the streams is changed so that the two streams can be 
discernable. 

<Realization of Control Procedure> 

Both the control procedures explained in the above-described embodiments using the flowcharts and the control pro- 
cedures of the functional components explained in the above-described embodiments satisfy the requirements for the 
45 "program invention" since the above-mentioned control procedures are realized concretely using the hardware resources 
and are the creation of a technical idea utilizing natural laws. 
[0224] ■ Production of Program of Present Invention 

The program of the present invention is an object program that can execute on a computer. The object program is 
composed of one or more program codes that cause the computerto execute each step in the flowchart or each procedure 

50 of the functional components. There are various types of program codes such as the native code of the processor, and 
JAVA ™ byte code. There are also various forms of realizing the steps of the program codes. For example, when each 
step can be realized by using an external function, the call statements for calling the external functions are used as the 
program codes. Program codes that realize one step may belong to different object programs. In the RISC processor 
in which the types of instructions are limited, each step of flowcharts may be realized by combining arithmetic operation 

55 instructions, logical operation instructions, branch instructions and the like. 

[0225] The program of the present invention can be produced as follows. First, the software developer writes, using 
a programming language, a source program that achieves each flowchart and functional component. In this writing, the 
software developer uses the class structure, variables, array variables, calls to external functions, and so on, which 
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conform to the sentence structure of the programming language he/she uses. 

[0226] The written source program is sent to the compiler as files. The compiler translates the source program and 
generates an object program. 

The translation performed by the compiler includes processes such as the sentence structure analysis, optimization, 
resource allocation, and code generation. In the sentence structure analysis, the characters and phrases, sentence 
structure, and meaning of the source program are analyzed and the source program is converted into an intermediate 
program. In the optimization, the intermediate program is subjected to such processes as the basic block setting, control 
flow analysis, and data flow analysis. In the resource allocation, to adapt to the instruction sets of the target processor, 
the variables in the intermediate program are allocated to the register or memory of the target processor. In the code 
generation, each intermediate instruction in the intermediate program is converted into a program code, and an object 
program is obtained. 

[0227] After the object program is generated, the programmer activates a linker. The linker allocates the memory 

spaces to the object programs and the related library programs, and links them together to generate a load module. The 

generated loadmodule is based on the presumption that it is read by the computer and causes the computer to execute 

the procedures indicated in the flowcharts and the procedures of the functional components. The program of the present 

invention can be produced in this way. 

■ Use of Program of Present Invention 

The program of the present invention can be used as follows. 

[0228] 

(i) Used as Embedded Program 

When the program of the present invention is used as an embedded program, the load module as the program is 
written into an instruction ROM, together with the Basic Input/Output System (BIOS) program and various pieces 
of middleware (operation systems). The program of the present invention is used as the control program of the 
reproduction apparatus 300 as the instruction ROM is embedded in the control unit and is executed by the CPU. 

[0229] 

(ii) Used as Application 

When the reproduction apparatus 300 is a hard-disk-embedded model, the Basic Input/Output System (BIOS) 
program is embedded in an instruction ROM, and various pieces of middleware (operation systems) are preinstalled 
in the hard disk. Also, a boot ROM for activating the system from the hard disk is provided in the reproduction 
apparatus 300. 

[0230] In this case, only the load module is supplied to the reproduction apparatus 300 via a transportable recording 
medium and/or a network, and is installed in the hard disk as one application. This enables the reproduction apparatus 
300 to perform the bootstrapping by the boot ROM to activate an operation system, and then causes the CPU to execute 
the installed load module as one application so that the program of the present application can be used. 
[0231] As described above, when the reproduction apparatus 300 is a hard-disk-embeddedmodel, the program of the 
present invention can be used as one application. Accordingly, it is possible to transfer, lend, or supply, via a network, 
the program of the present invention separately. 
<Controller 22> 

Components such as the controller 22 described above in the embodiments can be realized as one system LSI. 
[0232] The system LSI is obtained by implementing a bear chip on a high-density substrate and packaging them. The 
system LSI is also obtained by implementing a plurality of bear chips on a high-density substrate and packaging them, 
so that the plurality of bear chips have an outer appearance of one LSI (such a system LSI is called a multi-chip module). 
The system LSI has a QFP (Quad Flat Package) type and a PGA (Pin Grid Array) type. In the QFP-type system LSI, 
pins are attached to the four sides of the package. In the PGA-type system LSI, a lot of pins are attached to the entire 
bottom. 

[0233] These pins function as an interface with other circuits. The system LSI, which is connected with other circuits 
through such pins as an interface, plays a role as the core of the reproduction apparatus 300. 

The bear chip packaged in the system LSI includes a front-end unit, a back-end unit, and a digital processing unit. The 
front-end unit digitizes analog signals. The back-end unit converts digital data obtained through digital processes into 
the analog format and outputs the analog data. 

[0234] The internal-structure components shown in the above-described embodiments are implemented in the digital 
processing unit. 

As described above in "Used as Embedded Program", the load module as the program, the Basic Input/Output System 
(BIOS) program and various pieces of middleware (operation systems) are written into an instruction ROM. The major 
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improvement of the embodiments is achieved by the load module as the program. It is therefore possible to produce a 
system LSI of the present invention by packaging the instruction ROM, in which the load module as the program is 
stored, as the bear chip. 

[0235] In regards with a specific implementation method, it is preferable to use the SoC implementation or the SiP 
implementation. The SoC (System on Chip) implementation is a technology for printing a plurality of circuits onto a chip. 
The SiP (System in Package) implementation is a technology for packaging a plurality of circuits by resin or the like. 
Through these processes, a system LSI of the present invention can be produced based on the internal structure of the 
reproduction apparatus 300 described in each embodiment above. 

[0236] It should be noted here that although the term LSI is used here, it may be called IC, LSI, super LSI, ultra LSI 
or the like, depending on the level of integration. 

Further, part or all of the components of each reproduction apparatus may be achieved as one chip. The integrated 
circuit is not limited to the SoC implementation or the SiP implementation, but may be achieved by a dedicated circuit 
or a general purpose processor. It is also possible to achieve the integrated circuit by using the FPGA (Field Programmable 
Gate Array) that can be re-programmed after it is manufactured, or a reconfigurable processor that can reconfigure the 
connection and settings of the circuit cells inside the LSI. Furthermore, a technology for an integrated circuit that replaces 
the LSI may appear in the near future as the semiconductor technology improves or branches into another technologies. 
In that case, the new technology may be incorporated into the integration of the functional blocks constituting the present 
invention as described above. Such possible technologies include biotechnology. 

Industrial Applicability 

[0237] The recording medium and reproduction apparatus of the present invention can be mass-produced based on 
the internal structures of them shown in the embodiments above. As such, the recording medium and reproduction 
apparatus of the present invention has the industrial applicability. 

Claims 

1. A recording medium on which play list information and an audio stream are recorded, wherein 
the play list information contains main-path information and sub-path information, 

the main-path information specifies one among a plurality of digital streams as a main stream and defines a main 
reproduction section for the main stream, 

the sub-path information specifies another one among the plurality of digital streams as a sub stream and defines, 
for the sub stream, a sub reproduction section that is to be synchronized with the main reproduction section, 
the main stream includes a video stream and a primary audio stream multiplexed therein, 

the audio stream is specified as the sub stream and includes a plurality of packets which are each attached with a 
packet identifier that includes an upper field and a lower field, and 

the upper field indicates that the audio stream is a secondary audio stream whose reproduction output is able to be 
mixed with a reproduction output of the primary audio stream. 

2. The recording medium of Claim 1 , wherein 

the lower field of the packet identifier indicates a corresponding audio stream that is one among a plurality of 
secondary audio streams. 

3. The recording medium of Claim 1 , wherein 

the main-path information includes a stream table that has entries respectively corresponding to secondary audio 
streams, 

each entry in the stream table has a reference value of a packet identifier corresponding to a secondary audio stream 
of the entry, and 

in the stream table, the entries are arranged in an order in which the secondary audio streams respectively corre- 
sponding to the entries are selected. 

4. The recording medium of Claim 1 , wherein 

each of the entries respectively corresponding to the secondary audio streams is attached with combination infor- 
mation that indicates a stream number of a primary audio stream whose reproduction output is able to be mixed 
with a reproduction output of a secondary audio stream of the entry. 

5. The recording medium of Claim 1 , wherein 
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each secondary audio stream includes metadata that performs a control to either increase or decrease a gain of an 
audio output of a primary audio stream. 

. The recording medium of Claim 1 , wherein 

the sub-path information indicates a time point at which the metadata starts to perform the control of the gain, either 
as a start point of the sub reproduction section or as an end point of the sub reproduction section. 

. The recording medium of Claim 1 , wherein 

the sub-path information includes type information that indicates that an audio stream specified by the sub-path 
information itself is a secondary audio stream. 

. A reproduction apparatus for reproducing a main stream and a sub stream in accordance with play list information, 
a main reproduction section being defined for the main stream, and a sub reproduction section being defined for 
the sub stream, 

the play list information defines a reproduction section for each of a plurality of digital streams, and contains main- 
path information and sub-path information, 
the reproduction apparatus comprising: 

a first reading unit operable to read out, in accordance with the main-path information, one or more packets 
that constitute the main reproduction section of the main stream; 

a second reading unit operable to read out, in accordance with the sub-path information, one or more packets 
that constitute the sub reproduction section of the sub stream; 

a demultiplexing unit operable to refer to upper fields of packet identifiers respectively contained in the packets 

read out by the first reading unit and the second reading unit, and demultiplex the packets into first packets and 

second packets that respectively constitute a primary audio stream and a secondary audio stream; 

a first decoder operable to decode the first packets constituting the primary audio stream; 

a second decoder operable to decode the second packets constituting the secondary audio stream; and 

a mixing unit operable to mix decoding results, which is non-compressed digital data, of the first decoder and 

the second decoder. 

. The reproduction apparatus of Claim 8, wherein 

the main-path information includes a stream table that has entries respectively corresponding to secondary audio 
streams, 

each entry in the stream table has a reference value of a packet identifier corresponding to a secondary audio stream 
of the entry, 

the reproduction apparatus comprises: 

a procedure executing unit operable to select, according to a predetermined procedure, a secondary audio 
stream to be reproduced, among a plurality of secondary audio streams; and 

a status register operable to store a stream number of the selected secondary audio stream, wherein 
the demultiplexing performed by the demultiplexing unit includes comparing an upper field of a reference value 
of a packet identifier written in an entry included in the stream table that corresponds to the stream number, 
and an upper field of a packet identifier of the packet read out by the second reading unit, and outputting the 
packet constituting the secondary audio stream read out by the second reading unit to the second decoder if 
the comparison shows that the two upper fields match each other. 

0. The reproduction apparatus of Claim 9, wherein 

each entry is associated with combination information that indicates a stream number of a primary audio stream 
whose reproduction output is able to be mixed with a reproduction output of a secondary audio stream of the entry, 
the status register stores a stream number of a currently reproduced primary audio stream, 

the predetermined procedure for the procedure executing unit includes a judgment on whether or not a stream 
number of a primary audio stream stored in the status register matches the stream number of the primary audio 
stream indicated by the combination information, and 

the mixing unit performs the mixing if it is judged in the predetermined procedure that the stream number of the 
primary audio stream stored in the status register matches the stream number of the primary audio stream indicated 
by the combination information. 

1. The reproduction apparatus of Claim 8, wherein 
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the status register includes a register that stores profile information of the reproduction apparatus, 

the profile information indicates whether or not a mixing function is mounted in the reproduction apparatus, 

the mixing unit performs the mixing if the profile information indicates that the mixing function is mounted in the 

reproduction apparatus, and does not perform the mixing if the profile information indicates that the mixing function 

is not mounted in the reproduction apparatus. 

2. The reproduction apparatus of Claim 8 further comprising 

a converting unit operable to perform downmixing and/or downsampling on non-compressed digital data that is a 
result of decoding a primary audio stream, 

the mixing unit mixes an audio output of a secondary audio stream with a result of the downmixing and/or down- 
sampling by the converting unit. 

3. The reproduction apparatus of Claim 12, wherein 
each secondary audio stream includes metadata, and 

the converting unit performs a control to either increase or decrease a gain of an audio output of a primary audio 
stream, in accordance with the metadata. 

4. The reproduction apparatus of Claim 8, wherein 

the sub-path information indicates a time point at which the metadata starts to perform the control of the gain, either 
as a start point of the sub reproduction section or as an end point of the sub reproduction section. 

5. The reproduction apparatus of Claim 8 further comprising 

an auxiliary mixing unit operable to mix a click sound of a user operation with a reproduction output of a primary 
audio stream with which a reproduction output of a secondary audio stream has been mixed by the mixing unit. 

6. The reproduction apparatus of Claim 8 further comprising 

a transmission/reception unit operable to transmit, to a server apparatus, a pair of an identifier of an optical disc 
and an identifier of play list information recorded on the optical disc so that the server apparatus transmits the play 
list information based on the pair of identifiers, wherein 

the play list information has been received by the transmission/reception unit and written to an auxiliary recording 
medium. 

7. The reproduction apparatus of Claim 8, wherein 

when the second reading unit reads out a plurality of secondary audio streams, the demultiplexing unit refers to 
lower fields in addition to upper fields of packet identifiers of packets constituting the plurality of secondary audio 
streams, detects, among the packet identifiers of packets constituting the plurality of secondary audio streams, one 
or more packet identifiers whose upper field and lower field are predetermined values, and outputs one or more 
packets that correspond to the detected one or more packet identifiers, to the second decoder. 

8. The reproduction apparatus of Claim 8, wherein 

the main-path information includes a stream table that has entries respectively corresponding to secondary audio 
streams, 

each entry in the stream table has a reference value of a packet identifier corresponding to a secondary audio stream 
of the entry, and 

the reproduction apparatus comprises: 

a procedure executing unit operable to select, according to a predetermined procedure, a secondary audio 
stream to be reproduced, among a plurality of secondary audio streams; and 

a status register operable to store a stream number of the selected secondary audio stream, wherein 
the demultiplexing performed by the demultiplexing unit includes comparing an upper field and a lower field of 
a reference value of a packet identifier written in an entry included in the stream table that corresponds to the 
stream number, and an upper field and a lower field of a packet identifier of the packet read out by the second 
reading unit, and outputting the packet constituting the secondary audio stream read out by the second reading 
unit to the second decoder if the comparison shows that the two upper fields match each other. 

9. A program that causes a computer to execute a process of reproducing a main stream and a sub stream in accordance 
with play list information, a main reproduction section being defined for the main stream, and a sub reproduction 
section being defined for the sub stream, 
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the play list information defines a reproduction section for each of a plurality of digital streams, and contains main- 
path information and sub-path information 
the program comprising the steps of: 

reading out, in accordance with the main-path information, one or more packets that constitute the main repro- 
duction section of the main stream; 

reading out, in accordance with the sub-path information, one or more packets that constitute the sub reproduction 
section of the sub stream; 

referring to upper fields of packet identifiers respectively contained in the packets read out in the packet reading 

steps, and demultiplexing the packets into first packets and second packets that respectively constitute a primary 

audio stream and a secondary audio stream; 

decoding the first packets constituting the primary audio stream; 

decoding the secondpackets constituting the secondary audio stream; and 

mixing decoding results, which is non-compressed digital data, of the first decoder and the second decoder. 

20. A reproduction method of reproducing a main stream and a sub stream in accordance with play list information, a 
main reproduction section being defined for the main stream, and a sub reproduction section being defined for the 
sub stream, 

the play list information defines a reproduction section for each of a plurality of digital streams, and contains main- 
path information and sub-path information, 
the reproduction method comprising the steps of: 

reading out, in accordance with the main-path information, one or more packets that constitute the main repro- 
duction section of the main stream; 

reading out, in accordance with the sub-path information, one or more packets that constitute the sub reproduction 
section of the sub stream; 

referring to upper fields of packet identifiers respectively contained in the packets read out in the packet reading 

steps, and demultiplexing the packets into first packets and second packets that respectively constitute a primary 

audio stream and a secondary audio stream; 

decoding the first packets constituting the primary audio stream; 

decoding the second packets constituting the secondary audio stream; and 

mixing decoding results, which is non-compressed digital data, of the first decoder and the second decoder. 
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FIG. 27A PSR1: Primary Audio Stream Number 
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FIG. 27B PSR14: Secondary Audio Stream Number 
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FIG. 34 
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FIG. 38 
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FIG. 41 
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FIG. 47 
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FIG. 48 
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FIG. 49 
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